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Discontinuous dynamical systems arise in a large number of applications. In optimal control 
problems, open-loop bang-bang controllers switch discontinuously between extreme values of the 
inputs to generate minimum-time trajectories from the initial to the final states Thermostats 
implement on-off controllers to regulate room temperature When the temperature is above the 
desired value, the controller switches the cooling system on. Once the temperature reaches a preset 
value, the controller switches the cooling system off. The controller is therefore a discontinuous 
function of the room temperature. In nonsmooth mechanics, the motion of rigid bodies is subject 
to velocity jumps and force discontinuities as a result of friction and impact [3, 0]- In the robotic 
manipulation of objects by means of mechanical contact [1], discontinuities occur naturally from 
interaction with the environment. 

Discontinuities are also intentionally designed to achieve regulation and stabilization. Sliding 
mode control 0, 0] uses discontinuous feedback controllers for stabilization. The design procedure 
for sliding mode control begins by identifying a surface in the state space with the property that the 
dynamics of the system restricted to this surface are easily stabilizable. Feedback controllers are 
then synthesized on each side of the surface to steer the solutions of the system toward the surface. 
The resulting closed-loop system is discontinuous on the surface. In robotics it is of interest to 
induce emergent behavior in a swarm of robots by prescribing interaction rules among individual 
agents. Simple laws such as "move away from the nearest other robot or environmental boundary" 
give rise to discontinuous dynamical systems. For example, consider a robot placed in a corner 
of a room. On opposite sides of the bisector line between the two walls forming the corner, the 
"move away" law translates into different velocity vectors for the robot. The dynamical system is 
thus discontinuous along the bisector line. In optimization 0, [l^l) continuous-time algorithms that 
perform generalized gradient descent, which is a discontinuous function of the state, are used when 
the objective function is not smooth. In adaptive control [IH, switching algorithms are employed 
to select the most appropriate controller from a given finite family in order to enhance robustness, 
ensure boundedness of the estimates, and prevent the system from stepping into undesired regions 
of the state space. 



Many control systems cannot be stabilized by continuous state-dependent feedback. As a con- 
sequence, it is necessary to consider either time-dependent or discontinuous feedback. As an il- 
lustration, consider the one-dimensional system x = X{x,u) with an equilibrium at the origin for 



u = 0. When x > 0, we look for u such that X{x,u) < ("go left"). Likewise, when x < 0, 
we look for u such that X{x,u) > ("go right"). Together, these conditions can be stated as 
xX{x, u) < 0. When the control u appears nonlinearly in X, it may be the case that no continuous 
state-dependent feedback controller x i— > u{x) exists such that xX{x,u{x)) < for all x G M. The 
following example illustrates the above discussion. 



Example 1: System requiring discontinuous stabilization 



Consider the one-dimensional system [1/ 

± = x[{u - if - (x - l)][(n + if + (x - 2)]. (1) 

The shaded areas in Figure [1] represent the regions in the space {x,u) where xX{x,u) < 0. From 
the plot, it can be seen that there exists no continuous function x i— > u{x) defined on M whose 
graph belongs to the union of the shaded areas. Even control systems whose inputs appear linearly 
(see Example 12 ip may be subject to obstructions that preclude the existence of continuous state- 
dependent stabilizing feedback 12, 1^, 14]. ■ 



Numerous fundamental questions arise when dealing with discontinuous dynamical systems. The 
most basic question is the notion of a solution. For a discontinuous vector field, the existence of 
a continuously differentiable solution, that is, a continuously differentiable curve whose derivative 
follows the direction of the vector field, is not guaranteed. The following examples illustrate the 
difficulties that arise in defining solutions of discontinuous dynamical systems. 



Example 2: Brick on a frictional ramp 

Consider a brick sliding on a ramp [l^. As the brick moves, it experiences a friction force in the 
opposite direction (see Figure [2)^a)). During sliding, the Coulomb friction model states that the 
magnitude of the friction force is independent of the magnitude of the velocity and is equal to the 
normal contact force times the coefficient of friction. The application of this model to the sliding 
brick yields 

v{t) = g{sm 9) — ug{cos 6) sigxi{v{t)), (2) 

where v is the velocity of the brick, g is the acceleration due to gravity, > is the inclination of the 
ramp, v is the coefficient of friction, and sign(O) = 0. The right-hand side of ([2]) is a discontinuous 
function of v because of the presence of the sign function. Figure [2)[b) shows the phase plot of this 
system for several values of v. 

Depending on the magnitude of the friction force, experiments show that the brick stops and stays 
stopped. In other words, the brick attains t; = in finite time, and maintains w = 0. However, 
there is no continuously differentiable solution of ([2]) that exhibits this type of behavior. To see 
this, note that u = and = in (l2|) imply sm9 = 0, which contradicts > 0. In order to explain 
this type of physical evolution, we need to understand the discontinuity in ([2]) and expand our 
notion of solution beyond continuously differentiable solutions. ■ 
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Example 3: Nonsmooth harmonic oscillator 



Consider a unit mass subject to a discontinuous spring force. The spring does not exert any force 
when the mass is at the reference position x = 0. When the mass is displaced to the right, that is, 
X > 0, the spring exerts a constant negative force that pulls it back to the reference position. When 
the mass is displaced to the left, that is, x < 0, the spring exerts a constant positive force that 
pulls it back to the reference position. According to Newton's second law, the system evolution is 



described by [16t] 



X + sign(x) = 0. (3) 
By defining the state variables xi = x and X2 = x, ^ can be rewritten as 

±lit)=X2{t), (4) 

i2(i) = -sign(xi(t)), (5) 

which is a nonsmooth version of the classical harmonic oscillator. The phase portrait of this system 
is plotted in Figure [3]^a). 

It can be seen that (0,0) is the unique equilibrium of ([l])-([5]). By discretizing the equations of 
motion, we find that the trajectories of the system approach, as the time step is made smaller and 
smaller, the set of curves in Figure [3(b), which are the level sets of the function {xi,X2) ^ \xi\ + -^. 
These level sets are analogous to the level sets of the function (xj,X2) ^ xf + x^, which are 



the trajectories of the classical harmonic oscillator x + x = [17|]. However, the trajectories 
in Figure El^b) are not continuously differentiable along the X2-coordinate axis since the limiting 
velocity vectors from the right of the axis and from the left of the axis do not coincide. ■ 



Example 4: Move-away-from-nearest-neighbor interaction law 

Consider n nodes pi, . . . ,pn evolving in a square Q according to the interaction rule "move 
diametrically away from the nearest neighbor." Note that this rule is not defined when two nodes 
are located at the same point, that is, when the configuration is an element of 55" = {{pi, ■ ■ ■ ,Pn) S 
: Pi = pj for some i ^ j}. We thus consider only configurations that belong to Q^\SS. Next, 
we define the map that assigns to each node its nearest neighbor, where the nearest neighbor may be 
an element of the boundary bndry(Q) of the square. Note that the nearest neighbor of a node might 
not be unique, that is, more than one node can be located at the same (nearest) distance. Hence, 
we define the nearest-neighbor map M = (A/i, . . . ,Mn) '■ Q^\SS by arbitrarily selecting, for 

each i £ {1, . . . , n}, an element, 

Mi{pi, . . . ,p„) G argmin{||pi - q\\2 : q € bndry((5) U {pi, . . . ,Pn}\{Pi}}, 

where argmin stands for the minimizing value of q, and || • ||2 denotes the Euclidean norm. By 
definition, J\fi{pi, . . . ,Pn) 7^ Pi- For i G we thus define the move-away-from-nearest- 

neighbor interaction law 

. ^ p^{t) -Mi{pi{t),... ,pn{t)) 

||K(t)-M(pi(t),...,pJt))||2- ^ ' 
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Changes in the value of the nearest-neighbor map M induce discontinuities in the dynamical system. 
For instance, consider a node sufficiently close to a vertex of bndry((5) so that the closest neighbor 
to the node is an element of the boundary. Depending on how the node is positioned with respect 
to the bisector line passing through the vertex, the node computes different directions of motion, 
see Figure m for an illustration. 

To analyze the dynamical system ([6]), we need to understand how the discontinuities affect its 
evolution. Since each node moves away from its nearest neighbors, it is reasonable to expect that 
the nodes never run into each other, however, rigorous verification of this property requires a proof. 
We would also like to characterize the asymptotic behavior of the trajectories of the system In 
order to study these questions, we need to extend our notion of solution. ■ 



Beyond continuously different iable solutions 

Examples [2H] above can be described by a dynamical system of the form 

x{t) = X{x{t)), x{to) = xo, (7) 

where x G M*^, d is a positive integer, and X : M*^ — > M*^ is not necessarily continuous. We refer to 
a continuously differentiable solution t i— > of ([7]) as classical. Clearly, if X is continuous, then 
every solution is classical. Without loss of generality, we take to = 0- We consider only solutions 
that run forward in time. 

Examples [2]|3] illustrate the limitations of classical solutions, and confront us with the need to 
identify a suitable notion for solutions of ([7|). Unfortunately, there is not a unique answer to this 
question. Depending on the problem and objective at hand, different notions are appropriate. In 
this article, we restrict our attention to solutions that are absolutely continuous. Although not 
treated in detail in this article, it is also possible to consider solutions that admit discontinuities, 
and hence are not absolutely continuous. These solutions are discussed in "Solutions with Jumps." 

The function 7 : [a,b] — > M is absolutely continuous if, for all e € (0,oo), there exists 5 G (0, 00) 
such that, for each finite collection {(ai, 61), . . . , (a„, of disjoint open intervals contained in 
[a, b] with — Oj) < 6, it follows that 

n 

^h{bi) - l{ai)\ < e. 

i=l 

Equivalently [l2|, 7 is absolutely continuous if there exists a Lebesgue integrable function k : 
[a, 6] — > M such that 

j{t) = -f{a) + f K{s)ds, te[a,b]. 

J a 

Every absolutely continuous function is continuous. However, the converse is not true, since the 
function 7 : [—1, 1] — > M defined by "f{t) = tsin (j) for t / and 7(0) = is continuous, but not 
absolutely continuous. Moreover, every continuously differentiable function is absolutely continu- 
ous, but the converse is not true. For instance, the function 7 : [—1, 1] ^ M defined by 7(t) = |t| 
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is absolutely continuous but not continuously differentiable at 0. As this example suggests, every 
absolutely continuous function is differentiable almost everywhere. Finally, every locally Lipschitz 
function (see "Locally Lipschitz Functions") is absolutely continuous, but the converse is not true. 
For instance, the function 7 : [0, 1] — > M defined by 7(t) = y/t is absolutely continuous but not 
locally Lipschitz at 0. 



Caratheodory solutions [la] are a generalization of classical solutions. Roughly speaking, Caratheodory 
solutions are absolutely continuous curves that satisfy the integral version of the differential equa- 
tion d?]), that is, 

x{t) = x{to) + / X{x{s))ds, t > to, (8) 
Jto 

where the integral is the Lebesgue integral. By using the integral form ([8]), Caratheodory solutions 
relax the classical requirement that the solution must follow the direction of the vector field at all 
times, that is, the differential equation ([7]) need not be satisfied on a set of measure zero. As shown 
in this article, Caratheodory solutions exist for Example [3] but do not exist for examples [2] and HI 



Alternatively, Filippov solutions [l^ replace the differential equation ([7]) by a differential inclusion 
of the form 

x{t) e T{xit)), (9) 

where J- : M"' — > *B(M'^) and !B(M'^) denotes the collection of all subsets of M"^. Filippov solutions 
are absolutely continuous curves. At a given state x, instead of focusing on the value of the vector 
field at X, the idea behind Filippov solutions is to introduce a set of directions that are determined 
by the values of the vector field X in a neighborhood of x. Differential inclusions [H, ^ thus 



involve set-valued maps. Just as a standard map or function takes a point in its domain to a single 
point in another space, a set-valued map takes a point in its domain to a set of points in another 
space. A differential inclusion thus specifies that the state derivative belongs to a set of directions, 
rather than being a specific direction. This fiexibility is crucial for providing conditions on the 
discontinuous vector field under which Filippov solutions exist. This solution notion plays a key 
role in many of the applications mentioned above, including sliding mode control and mechanics 
with Coulomb-like friction. 



Unfortunately, the obstructions to continuous stabilization illustrated in Example [T] also hold 21 



22I ] for Filippov solutions. Sample- and- hold solutions [231 ] . which are also absolutely continuous 



curves, turn out to be the appropriate notion for circumventing these obstructions [24 . l25l . \2t 
"Additional Solution Notions for Discontinuous Systems" describes additional solution notions for 
discontinuous systems. In this article, we focus on Caratheodory, Filippov, and sample-and-hold 
solutions. 



Existence, uniqueness, and stability of solutions 



In addition to the notion of solution, we consider existence and uniqueness of solutions as well 
as stability. For ordinary differential equations, it is well known that continuity of the vector field 
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does not guarantee uniqueness. Not surprisingly, no matter what notion of solution is chosen for 
a discontinuous vector field, nonuniqueness can occur. We thus provide sufficient conditions for 
uniqueness. We also present results specifically tailored to piecewise continuous vector fields and 
differential inclusions. 

The lack of uniqueness of solutions must be considered when we try to establish properties such as 
local stability. This issue is reflected in the use of the adjectives weak and strong. The word "weak" 
is used when a property is satisfied by at least one solution starting from each initial condition. 
On the other hand, "strong" is used when a property is satisfied by all solutions starting from each 
initial condition. Therefore, for example, "weakly stable equilibrium point" means that at least 
one solution starting close to the equilibrium point remains close to it, whereas "strongly stable 
equilibrium point" means that all solutions starting close to the equilibrium point remain close to 
it. For detailed definitions, see 
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We present weak and strong stability results for discontinuous dynamical systems and differential 
inclusions. As suggested by Example [3l smooth Lyapunov functions do not suffice to analyze the 
stability of discontinuous systems. This fact leads naturally to the subject of nonsmooth analysis. 
In particular, we pay special attention to the generalized gradient of a locally Lipschitz function 



and the proximal subdifferential of a lower semicontinuous function 2J]. Building on these notions, 
weak and strong monotonicity properties of candidate Lyapunov functions can be established along 
the solutions of discontinuous dynamical systems. These results are used to provide generalizations 
of Lyapunov stability theorems and the invariance principle, which help us study the stability of 
solutions. To illustrate the applicability of these results, we discuss in detail a class of nonsmooth 
gradient flows. 

There are two ways to apply the stability results presented here to control systems. The first way 
is to choose a specific input function and consider the resulting dynamical system. The second way 
is to associate with the control system the set-valued map that assigns each state to the set of all 
vectors generated by the allowable inputs, and consider the resulting differential inclusion. Rather 
than focusing on a particular input, this viewpoint allows us to consider the full range of trajectories 
of the control system. To analyze the stability of the control system under this approach, we can 
use the nonsmooth tools developed for differential inclusions. We explore this idea in detail. 

Given the large body of work on discontinuous systems, our aim is to provide a clear exposition 
of a few useful and central results. "Additional Topics on Discontinuous Systems and Differential 
Inclusions" briefly discusses issues that are not considered in the main exposition. 



Organization of this article 

We start by reviewing basic results on the existence and uniqueness of classical, that is, continu- 
ously differentiable, solutions of ordinary differential equations. We also present several examples 
in which the vector field fails to satisfy the standard smoothness properties. We then introduce 
various notions of solution for discontinuous systems, discuss existence and uniqueness results, and 
present useful tools for analysis. In preparation for the statement of stability results, we introduce 
the generalized gradient and proximal subdifferential from nonsmooth analysis, and present various 
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tools for their explicit computation. Then, we develop analysis results to characterize the stabil- 
ity and asymptotic convergence properties of the solutions of discontinuous dynamical systems. 
We illustrate these nonsmooth stability results by means of examples, paying special attention to 
gradient systems. Throughout the discussion, we interchangeably use "differential equation," "dy- 
namical system," and "vector field." For reference, "Index of Symbols" summarizes the notation 
used throughout. 

To simplify the presentation, we have chosen to restrict our attention to time-invariant vector 
fields, although most of the development can be adapted to the time-varying setting. We briefly 
discuss time-varying systems in "Caratheodory Conditions for Time-varying Vector Fields" and 
"Caratheodory Solutions of Differential Inclusions." Likewise, for simplicity, we mostly consider 
vector fields defined over the whole Euclidean space, although the exposition can be carried out in 
more general settings such as open and connected subsets of the Euclidean space. 



Existence and Uniqueness for Ordinary Differential Equations 

In this section, we review basic results on existence and uniqueness of classical solutions for 
ordinary differential equations. We also present examples that do not satisfy the hypotheses of 
these results but nevertheless exhibit existence and uniqueness of classical solutions, as well as 
examples that do not possess such desirable properties. 



Existence of classical solutions 

Consider the differential equation 



x{t) = X{x{t)), (10) 



where X : M'^ — > M'^ is a vector field. The point Xe G M'^ is an equilibrium of (jlOp if = X{xe)- 
A classical solution of (jlOh on [0,ti] is a continuously differentiable map x : [0,ti] M"' that 
satisfies (|10p . Note that, without loss of generality, we consider only solutions that start at time 
0. Usually, we refer to t x{t) as a classical solution with initial condition x(0). We sometimes 
write the initial condition as xq instead of x{0). The solution t x{t) is maximal if it cannot be 
extended forward in time, that is, if i i-^ x{t) is not the result of the truncation of another solution 
with a larger interval of definition. Note that the interval of definition of a maximal solution is 
either of the form [0, T), where T > 0, or [0, oo). 

Continuity of the vector field suffices to guarantee the existence of classical solutions, as stated 



by Peano's theorem [27[. 



Proposition 1. Let X : — > M.'^ be continuous. Then, for all xq G M.'^, there exists a 
classical solution of (llOp with initial condition x{0) = xq. 

The following example shows that, if the vector field is discontinuous, then classical solutions 
of ([To]) might not exist. 
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Example 5: Discontinuous vector field with nonexistence of classical solutions 



Consider the vector field X : M ^ M defined by 

xix)=h^ (11) 

[1, X < 0, 

which is discontinuous at (see Figure [5]^ a)). Suppose that there exists a continuously differentiable 
function x : [0,ti] M such that x{t) = X{x{t)) and x(0) = 0. Then i(0) = X{x{0)) = X{0) = 1, 
which implies that, for all positive t sufficiently small, x{t) > and hence x{t) = X{x{t)) = — 1, 
which contradicts the fact that 1 1— > x{t) is continuous. Hence, no classical solution starting from 
exists. ■ 

In contrast to Example [5l the following example shows that the lack of continuity of the vector 
field does not preclude the existence of classical solutions. 

Example 6: Discontinuous vector field with existence of classical solutions 

Consider the vector field X : M — > M, 



X{x) = — sign(x) 



-1, X > 0, 

0, X = 0, (12) 

1, X <0, 



which is discontinuous at (see Figure [5l^b)). If x(0) > 0, then the maximal solution x : [0, x(0)) 
M is x{t) = x{0) — t, whereas, if x(0) < 0, then the maximal solution x : [0, — x(0)) — > M is 
x(t) = x{0) + 1. Finally, if x(0) = 0, then the maximal solution x : [0, oo) — > M is x{t) = 0. Hence 
the associated dynamical system x{t) = X{x{t)) has a classical solution starting from every initial 
condition. Although the vector fields (jlip and (jl2p are identical except for the value at 0, the 
existence of classical solutions is surprisingly different. ■ 



Uniqueness of classical solutions 

Uniqueness of classical solutions of the differential equation (jlOp means that every pair of solutions 
with the same initial condition coincide on the intersection of their intervals of existence. In other 
words, if xi : [0, ti] — > M'^ and X2 : [0, — ^ 1^*^ are classical solutions of (jlOp with xi(0) = X2(0), 
then xi{t) = X2{t) for all t G [0,ti] Pi [0,t2] = [0, min{ti, t2}]- Equivalently, we say that there exists 
a unique maximal solution starting from each initial condition. 



Uniqueness of classical solutions is guaranteed under a wide variety of conditions. The book 
for instance, is devoted to collecting various uniqueness criteria. Here, we focus on a uniqueness 
criterion based on one-sided Lipschitzness. The vector field X : M'^ ^ is one-sided Lipschitz on 
[/ C M'^ if there exists L > such that, for all y, y' G U, 

[Xiy)-Xiy')fiy-y')<L\\y-y'\\l (13) 
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This property is one-sided because it imposes a requirement on X only when the angle between the 
two vectors on the left-hand side of (jl3p is between and 180 degrees. The following result, given 
in [2^, uses this property to provide a sufficient condition for uniqueness. 

Proposition 2. Let X ^ he continuous. Assume that, for all x S M'^, there 
exists e > such that X is one-sided Lipschitz on B{x,e). Then, for all xq S W^, there exists a 
unique classical solution of (fTOll with initial condition x(0) = xq. 

Every vector field that is locally Lipschitz at x (see "Locally Lipschitz Functions") satisfies the 
one-sided Lipschitz condition on a neighborhood of x, but the converse is not true. For example, the 
vector field X : M — > R defined by X[Q) = and X{x) = 2;log(|x|) for x 7^ is one-sided Lipschitz 
on a neighborhood of 0, but is not locally Lipschitz at 0. Furthermore, a one-sided Lipschitz vector 
field can be discontinuous. For example, the discontinuous vector field (112p is one-sided Lipschitz 
on a neighborhood of 0. 

The following result is an immediate consequence of Proposition [21 

Corollary 1. Let X '.W^ he locally Lipschitz. Then, for all xq G R*^, there exists 

a unique classical solution of (llOp with initial condition x(0) = xq. 

Although local Lipschitzness is typically invoked as in Corollary[T]to guarantee uniqueness, Propo- 
sition [2] shows that uniqueness is guaranteed under weaker hypotheses. The following example shows 
that, if the hypotheses of Proposition [2] are not satisfied, then solutions might not be unique. 



Example 7: Continuous, not one-sided Lipschitz vector field with nonunique classical 
solutions 



Consider the vector field X : R — > R defined by 

X{x) = ^/\^\. (14) 

This vector field is continuous everywhere, locally Lipschitz on R\{0} (see Figure [5]^c) ) , but is 
neither locally Lipschitz at nor one-sided Lipschitz on any neighborhood of 0. The associated 
dynamical system x{t) = X{x[t)) has infinitely many maximal solutions starting from 0, namely, 
for all a > 0, : [0, 00) — > R, where 




Xa{t) = 

and X : [0, 00) — > R, where x{t) = 0. ■ 

However, the following example shows that a differential equation can possess a unique classical 
solution even when the hypotheses of Proposition [2] are not satisfied. 
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Example 8: Continuous, not one-sided Lipschitz vector field with unique classical 
solutions 

Consider the vector field X : M ^ M defined by 



This vector field is continuous everywhere, and locally Lipschitz on ]R\{0} (see Figure [5]^d)). How- 
ever, X is not locally Lipschitz at nor one-sided Lipschitz on any neighborhood of 0. Nevertheless, 
the associated dynamical system x{t) = X{x{t)) has a unique solution starting from each initial 
condition. If x(0) > 0, then the maximal solution x : [0,oo) — > M is x{t) = exp(log x(0) exp(— t)), 
whereas, if x(0) < 0, then the maximal solution x : [0, oo) ^ M is x{t) = — exp(log(— x(0)) exp(t)). 
Finally, if x(0) = 0, then the maximal solution x : [0, oo) — > M is x{t) = 0. ■ 

Note that Proposition [2] assumes that the vector field is continuous. However, the discontinuous 
vector field (I12p in Example[6]is one-sided Lipschitz in a neighborhood of and, indeed, has a unique 
classical solution starting from each initial condition. This observation suggests that discontinuous 
systems are not necessarily more complicated or less "well-behaved" than continuous systems. The 
continuous system (jl4p in Example [7] does not have a unique classical solution starting from each 
initial condition, whereas the discontinuous system (jl2p in Example [6] does. A natural question to 
ask is under what conditions does a discontinuous vector field have a unique solution starting from 
each initial condition. Of course, the answer to this question relies on the notion of solution itself. 
We explore these questions in the next section. 

Notions of Solution for Discontinuous Dynamical Systems 

The above discussion shows that the classical notion of solution is too restrictive when consid- 
ering a discontinuous vector field. We thus explore alternative notions of solution to reconcile 
this mismatch. To address the discontinuities of the differential equation (llOp . we first relax the 
requirement that solutions follow the direction specified by the vector field at all times. The pre- 
cise mathematical notion corresponding to this idea is that of Caratheodory solutions, which we 
introduce next. 

Caratheodory solutions 

A Caratheodory solution of (llOp defined on [0, ti] C M is an absolutely continuous map x : 
[0,ti] — > M'^ that satisfies (jlOp for almost all t G [0, ii] (in the sense of Lebesgue measure). In 
other words, a Caratheodory solution follows the direction specified by the vector field except for 
a set of time instants that has measure zero. Equivalently, Caratheodory solutions are absolutely 




(15) 
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continuous functions that solve the integral version of (jlOp . that is, 

x{t)=x{0)+[ X{x{s))ds. (16) 
Jo 

Of course, every classical solution is also a Caratheodory solution. 



Example 9: System with Caratheodory solutions and no classical solutions 

The vector field X : M ^ M defined by 



Xix) 



1, X > 0, 
,-1, a;<0. 



is discontinuous at 0. The associated dynamical system x{t) = X{x{t)) does not have a classical 
solution starting from 0. However, this system has two Caratheodory solutions starting from 0, 
namely, xi : [0, oo) M, where xi{t) = t, and X2 : [0, oo) — > R, where X2{t) = —t. Note that 
both xi and X2 violate the differential equation only at t = 0, that is, xi(0) 7^ X{xi{0)) and 

±2(0) / X(X2(0)). ■ 



Example [3] revisited: Existence of Caratheodory solutions for the nonsmooth harmonic 
oscillator 

The nonsmooth harmonic oscillator in Example [3] does not possess a classical solution starting 
from any initial condition on the X2-axis. However, the closed level sets depicted in Figure E^b) , 
when traversed clockwise, are Caratheodory solutions. ■ 

Unfortunately, the good news is quickly over since it is easy to find examples of discontinuous 
dynamical systems that do not admit Caratheodory solutions. For example, the physical mo- 
tions observed in Example El where the brick slides for a while and then remains stopped, are 
not Caratheodory solutions. Furthermore, the discontinuous vector field (jlip does not admit a 
Caratheodory solution starting from 0. Finally, the move-away-from-nearest-neighbor interaction 
law in Example U] is yet another example where Caratheodory solutions do not exist, as we show 
next. 



Example [4] revisited: Nonexistence of Caratheodory solutions for the move-away- from- 
nearest-neighbor interaction law 

Consider one agent moving in the square [-1,1] 2 c under the move-away- from-nearest- 
neighbor interaction law described in Example HI Since no other agent is present in the square, the 
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agent moves away from the nearest polygonal boundary, according to the vector field 



X{xi,X2) 



(-1,0), -Xi < X2< Xl, 

(0,1), X2<Xi<-X2, ^^^^ 

(1,0), Xl < X2 < -Xl, 

(0, -1), -X2 < Xl < X2- 



Since the move-away-from-nearest-neighbor interaction law takes multiple values on the diagonals 
{(a,±a) G [-1,1]2 : a G [-1, 1]} of the square, we choose one of these values in the definition (|17p 
of X. The phase portrait in Figure O^a) shows that the vector field X is discontinuous on the 
diagonals. 

The dynamical system x{t) = X{x{t)) has no Caratheodory solution if and only if the initial con- 
dition belongs to the diagonals. This fact can be justified as follows. Consider the four open regions 
of the square separated by the diagonals, see Figure El^a) . On the one hand, if the initial condition 
belongs to one of these regions, then the dynamical system has a classical solution; depending 
on which region the initial condition belongs to, the agent moves either vertically or horizontally 
toward the diagonals. On the other hand, on the diagonals of the square, X pushes trajectories 
outward, whereas, outside the diagonals, X pushes trajectories inward. Therefore, if the initial 
condition belongs to the diagonals, then the only candidate trajectory for a Caratheodory solution 
is a trajectory that moves along the diagonals. From the definition (|16p of Caratheodory solution, 
it is clear that a trajectory that moves along the diagonals of the square is not a Caratheodory 
solution. We show later that this trajectory is instead a Filippov solution. ■ 



Sufficient conditions for the existence of Caratheodory solutions 



Conditions under which Caratheodory solutions exist are discussed in "Caratheodory Conditions 
for Time-varying Vector Fields." For time-invariant vector fields, the Caratheodory conditions 
given by Proposition ISII specialize to continuity of the vector field. This requirement provides 
no improvement over Proposition [U which guarantees the existence of classical solutions under 
continuity. 

Therefore, it is of interest to determine conditions for the existence of Caratheodory solutions 
specifically tailored to time-invariant vector fields 
fields are considered in 29|]. A vector field X : 

6 G (0,oo) such that, for every x G M"^ with X{x) ^ and every sequence {xnjneN C 
x„ — > X and 



For example, directionally continuous vector 
> M'' is directionally continuous if there exists 

with 



X{x) 



X\\2 



X x) 



<S, n G N, 



(18) 



it follows that X(x„) X{x). If the vector field X is directionally continuous, then, for all xq G 
there exists a Caratheodory solution of (jlOp with initial condition x(0) = xq. 



Patchy vector fields [30|] are another class of time- invariant, discontinuous vector fields that have 
Caratheodory solutions. Additional conditions for the existence and uniqueness of Caratheodory 
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conditions can be found in 31, 3^, 33]. Beyond differential equations, Caratheodory solutions can 
also be defined for differential inclusions, as explained in "Set-valued Maps" and "Caratheodory 
Solutions of Differential Inclusions." 



Filippov solutions 

The above discussion shows that the relaxation of the value of the vector field on a set of times 
of measure zero in the definition of Caratheodory solution is not always sufficient to guarantee that 
such solutions exist. Due to the discontinuity of the vector field, its value can exhibit significant 
variations arbitrarily close to a given point, and this mismatch might make it impossible to construct 
a Caratheodory solution. 

What if, instead of focusing on the value of the vector field at individual points, we consider how 
the vector field looks like around each point? The idea of looking at a neighborhood of each point 



solution [3J| and Sentis solution 



is at the core of the notion of Filippov solution 18|] . Closely related notions are those of Krasovskii 



31 



The mathematical framework for formalizing this neighborhood idea uses set-valued maps. The 
idea is to associate a set-valued map to X : M"^ ^ R'^' by looking at the neighboring values of X 
around each point. Specifically, for x G W^, the vector field X is evaluated at the points belonging 
to B{x,6), which is the open ball centered at x with radius 5 > 0. We examine the effect of 6 
approaching by performing this evaluation for smaller and smaller 5. For additional fiexibility, 
we exclude an arbitrary set of measure zero in B(x, 6) when evaluating X, so that the outcome is 
the same for two vector fields that differ on a set of measure zero. 

Mathematically, the above procedure can be summarized as follows. Let ?B(M'^) denote the 
collection of subsets of M'^. For X :R'^ ^ W^, define the Filippov set-valued map F[X] : M'^ 53(M'^) 

by 

F[X]{x) = f] Pi m{X{B{x,5)\S)}, xeM.'^. (19) 
5>0/i(5)=0 

In p^ . CO denotes convex closure, and /i denotes Lebesgue measure. Because of the way the 
Filippov set-valued map is defined, the value of F[X] at a point x is independent of the value of 
the vector field X at x. 



Examples [5] and [6] revisited: Filippov set- valued map of the sign function 



Let us compute the Filippov set-valued map for the vector fields (jlip and (jl2p . Since both vector 
fields differ only at 0, which is a set of measure zero, their associated Filippov set-valued maps are 
identical and equal to F[X] : M 53 (M), where 



F[X]ix) 



-1,1], 



X > 0, 
x = 0, 
X <0. 



(20) 



13 



Note that this Fihppov set-valued map is multiple-valued only at the point of discontinuity of the 
vector field. This observation is valid for all vector fields. ■ 



We are now ready to handle the discontinuities of the vector field X by using the Filippov set- 
valued map of X. We replace the differential equation x{t) = X{x{t)) by the differential inclusion 

xit) € F[X]ix{t)). (21) 

A Filippov solution of (|10p on [0, ti] C M is an absolutely continuous map x : [0, ti] — > R"^ that 
satisfies (I2ip for almost all t £ [0,ti]. Equivalently, a Filippov solution of (|10p is a Caratheodory 
solution of the differential inclusion (j2ip . see "Caratheodory Solutions of Differential Inclusions." 

Because of the way the Filippov set-valued map is defined, a vector field that differs from X on 
a set of measure zero has the same Filippov set-valued map, and hence the same set of solutions. 
The next result establishes mild conditions under which Filippov solutions exist 3, 19]. 



Proposition 3. Let X : M'^ — > M'^ be measurable and locally essentially bounded, that 
is, bounded on a bounded neighborhood of every point, excluding sets of measure zero. Then, for all 
xq G , there exists a Filippov solution of (jlOp with initial condition x(0) = xq. 

In Proposition [31 the hypotheses on the vector field imply that the associated Filippov set- valued 
map satisfies all of the hypotheses of Proposition ISII (see "Caratheodory Solutions of Differential 
Inclusions"), which in turn guarantees the existence of Filippov solutions. 



Examples [5] and [6] revisited: Existence of Filippov solutions for the sign function 

The application of Proposition[3]to the bounded vector fields in examples[5]and[6]guarantees that a 
Filippov solution of (llOp exists for both examples starting from each initial condition. Furthermore, 
since the vector fields (jlip and (jl2p have the same Filippov set-valued map (j20p . examples [5] and [6] 
have the same maximal Filippov solutions. If x(0) > 0, then the maximal solution x : [0, cxo) M 
is 



x(0)-t, t<x(0), 
0, t > x(0). 



x{t) 

whereas, if x(0) < 0, then the maximal solution x : [0, oo) ^ M is 

x{t) 



x{<d) + t, t<-x(0), 
0, t > -x(0). 



Finally, if x(0) = 0, then the maximal solution x : [0, oo) ^ M is x{t) = 0. ■ 

Similar computations can be made for the move-away-from-nearest-neighbor interaction law in 
Example H] to show that Filippov solutions exist starting from each initial condition, as we show 
next. 
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Example |4] revisited: Filippov solutions for the move-away-from-nearest-neighbor in- 
teraction law 



Consider again the discontinuous vector field for one agent moving in the square [—1,1]^ C 
under the move-away-from-nearest-neighbor interaction law described in Example [H The corre- 
sponding set- valued map F[X] : [—1, 1]^ 



F[X]{xi,X2) 



{{yi,y2) G 
{(-1,0)}, 
{{yi,y2) G 

{(0,1)}, 

{{yi,y2) G 

{(1,0)}, 

{(.yi,y2) G 
{(0,-1)}, 
,{(yi,y2) G 



> *B(M^) is given by 

\yi + y2\ < 1, \yi - y2\ < i}, 
yi + 2/2 = -i,yi G [-1,0]}, 
yi-y2 = -i,yi g [-1,0]}, 
yi + ?/2 = i,yi G [0,1]}, 
yi-y2 = i,yi g [0, 1]}, 



(xi,X2) = (0,0), 
— Xi < X2 < Xl, 
< X2 = Xl, 
X2 < Xl < -X2, 
< -Xl = X2, 
Xl < X2 < —Xl, 
X2 = Xl < 0, 
-X2 < Xl < X2, 
< Xl = —X2- 



(22) 



Since X is bounded, it follows from Proposition [3] that a Filippov solution exists starting from each 
initial condition, see Figure [6l^b). In particular, each solution starting from a point on a diagonal 
is a straight line flowing along the diagonal itself and reaching (0,0). For example, the maximal 
solution X : [0, oo) — > starting from {a, a) G is given by 



x{t) 



(a — I sign(a)t, a — ^ sign(a)t), t < \a\, 
(0,0), t>\a\. 



(23) 



Note that the solution slides along the diagonals, following a convex combination of the limiting 
values of X around the diagonals, rather than the direction specified by X itself. We study this type 
of behavior in more detail in the section "Piecewise continuous vector fields and sliding motions." 



Relationship between Caratheodory and Filippov solutions 

In general, Caratheodory and Filippov solutions are not related. A vector field for which both 
notions of solution exist but Filippov solutions are not Caratheodory solutions is given in [s^] . The 
following is an example of the opposite case. 

Example 10: Vector field with a Caratheodory solution that is not a Filippov solution 

Consider the vector field X : M — > R given by 
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x/0, 
x = Q. 



(24) 



The dynamical system (jlOp has two Caratheodory solutions starting from 0, namely, xi : [0, oo) 
R, where xi{t) = 0, and X2 : [0, cxo) M, where X2{t) = t. However, the associated Filippov 
set-valued map F[X] : M 53(IR) is F[X]{x) = {1}, and hence t i-^ X2(t) is the unique Filippov 
solution starting from 0. ■ 

On a related note, Caratheodory solutions are always Krasovskii solutions (see "Additional So- 
lution Notions for Discontinuous Systems") but the converse is not true [l6| . 

Computing the Filippov set-valued map 



Computing the Filippov set-valued map can be a daunting task. A calculus is developed in [36|| to 
simplify this calculation. We summarize some useful facts below. Note that the Filippov set-valued 
map can also be constructed for maps of the form X : M"' — > R™, where d and m are not necessarily 
equal. 

Consistency. If X -.W^ —>■ is continuous at x G M'^, then 

F[X]{x) = {X{x)}. (25) 

Sum rule. If Xi, X2 : M'^ ^ is locally bounded at x G M.'^, then 

+ X2]{x) C F[Xi]{x) + F[X2]{x). (26) 

Moreover, if either Xi or X2 is continuous at x, then equality holds. 

Product rule. If Xi:R'^ ^ R"" and X2 : M'^ ^ M" are locally bounded at x G M'', then 

F[{Xi,X2f]{x) C F[Xi]{x) X F[X2]{x). (27) 

Moreover, if either Xi or X2 is continuous at x, then equality holds. 

Chain rule. If y : M'^ ^ is continuously differentiable at x G M*^ with Jacobian rank n, and 
X : M" ^ is locally bounded at Y{x) G M", then 

F[X oY]{x) = F[X]{Y{x)). (28) 

Matrix transformation rule. If X : M'' ^ is locally bounded at x G M'^ and Z :R'^ ^ M'^^™ 
is continuous at x G M*^, then 

F[ZX]{x) = Z{x)F[X]{x). (29) 



Piecewise continuous vector fields and sliding motions 



In this section we consider vector fields that are continuous everywhere except on a surface of the 
state space. Consider, for instance, two continuous dynamical systems, one per side of the surface, 
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glued together to give rise to a discontinuous dynamical system. Here, we analyze the properties 
of the Filippov solutions of this type of systems. 

The vector field X : M'^ ^ M"^ is piecewise continuous if there exists a finite collection of disjoint, 
open, and connected sets Pi, . . . ,I?m C M"' whose closures cover W^, that is, R"' = U^^^Pfc, such 
that, for all k = 1, . . . ,m, the vector field X is continuous on Dk. We further assume that the 
restriction of X to admits a continuous extension to the closure T>k, which we denote by ^|^- 
Every point of discontinuity of X must therefore belong to the union of the boundaries of the sets 
Vi, . . . ,Pm- Let us denote by Sx C bndry(Pi) U . . . U bndry(Pm) the set of points where X is 
discontinuous. Note that Sx has measure zero. 

The Filippov set-valued map of a piecewise continuous vector field X is given by the expression 

F[X]{x) = co{ lim X{xi) : x, ^ x , Xi (f, Sx}- (30) 

i— too 

This set- valued map can be computed as follows. At points of continuity of X, that is, for x Sx^ 
the consistency property (p5]) implies that F[X](x) = At points of discontinuity of X, 

that is, for x G Sx-, FX^^^^^ is a convex polyhedron in M'^ of the form 

F{X\{x) = co{X^{x) : X G bndry(Pfe)}. (31) 
As an illustration, let us revisit examples [2][31 



Examples [2}|4] revisited: Computation of the Filippov set-valued map and Filippov 
solutions 

The vector field of the sliding brick in Example [5] is piecewise continuous with Pi = {t; G M : 
V < 0} and P2 = {i; G K : t> > 0}. Note that the restriction of the vector field to Pi can be 
continuously extended to Pi by setting X|^(0) = g{s\n6 + vcosO). Likewise, the restriction of 

the vector field to P2 can be continuously extended to P2 by setting X|^(0) = g{sm9 — i^cosO). 
Therefore, the associated Filippov set- valued map F[X] : M — > *B(M) is given by 



F[X]iv) 



{(7(sin0 — z/cos6')}, v > 0, 

{g{sme - di^cosO) : dG[-l,l]}, v = 0, 
^{^(sin^ -|- z^cos^)}, f < 0, 



which is singleton- valued for all v Sx = {0}, and a closed segment at v = 0. If the friction 
coefficient v is large enough, that is, satisfies > tan^, then F[3;](t>) < if v > and F[x]{v) > 
if < 0. Therefore, the Filippov solutions that start with an initial positive velocity v eventually 
reach v = and remain at 0. This fact precisely corresponds to the observed physical motions for 
Example [21 where the brick slides for a while and then remains stopped. This example shows that 
Filippov solutions have physical significance. 

The vector field X : — > for the nonsmooth harmonic oscillator in Example [3] is continuous 
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on each of the half planes {1)1,1)2}, with 



Pi = {{xi,X2) G : XI < 0}, 
V2 = {ixi,X2) G : XI > 0}, 

and discontinuous on Sx = {(0,X2) : X2 G M}. Therefore, X is piecewise continuous. Its Filippov 
set- valued map F[X] iR^ ^^{M?) is given by 



F[X]{xuX2) 



{(x2,-sign(xi))}, XI / 0, 
{X2} X [-1, 1], xi = 0. 



Therefore, the closed level sets depicted in Figure ^h), when traversed clockwise, are Filippov 
solutions. 

The discontinuous vector field X : [—1,1]'^ — > for one agent moving in the square [—1, 1]^ C M? 
under the move-away-from-nearest-neighbor interaction law described in Example U] is piecewise 
continuous, with 



V^={{xi,X2) G [-1,1]' 

= {(X1,X2) G [-1,1]' 
V3 = {{XI,X2) G [-1,1]2 
V^ = {{xi,X2) G [-1,1]2 



-Xl < X2 < Xl}, 
X2 < Xi < -X2}, 
Xl < X2 < -Xl}, 
-X2 < Xl < X2}. 



Its Filippov set-valued map, described in (p2]) . maps points outside the diagonals Sx = {{a,±a) G 
[—1,1]^ : a G [—1,1]} to singletons, maps points in 5x\{(0,0)} to closed segments, and maps 
(0, 0) to a square polygon. Several Filippov solutions starting from various initial conditions are 
plotted in Figure ID^b). ■ 

Let us now discuss what happens on the points of discontinuity of the vector field X : R'^ ^ R'^. 
Suppose that x G Sx belongs to just two boundary sets, that is, x G bndry(Pj) n bndry('Dj), for 
some distinct i,j G {1, . . . ,m}, but x bndry(2?fc), for A; G {1, . . . ,m}\{i,j}. In this case, 

F[X](x)=co{X|^(x),X|^(x)}. 

We consider three possibilities. First, if all of the vectors belonging to -F[X](x) point into Di, then 
a Filippov solution that reaches Sx at x continues its motion in Vi (see Figure [8|^a)). Likewise, if 
all of the vectors belonging to -F[X](x) point into T>j, then a Filippov solution that reaches Sx at 
X continues its motion in Dj (see Figure [S]^b)). Finally, if a vector belonging to -F[X](x) is tangent 
to Sx, then either all Filippov solutions that start at x leave Sx immediately (see Figure [8)^c) ) , 
or there exist Filippov solutions that reach the set Sx at x, and remain in Sx afterward (see 
Figure Hd)). 

The last kind of trajectories are called sliding motions, since they slide along the boundaries of 
the sets where the vector field is continuous. This type of behavior is illustrated for Example H] 
in ()23p . Sliding motions can also occur along points belonging to the intersection of more than 
two sets in Vi, . . . The theory of sliding mode control builds on the existence of this type 
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of trajectories to design stabilizing feedback controllers. These controllers induce sliding surfaces 
with the desired properties so that the closed-loop system is stable 0, Q] • 

The solutions of piecewise continuous vector fields appear frequently in state-dependent switching 



dynamical systems [37l.l38l|. Consider, for instance, the case of two dynamical systems with the same 
unstable equilibrium. By identifying an appropriate switching surface between the two systems, it 
is often possible to synthesize a discontinuous dynamical system for which the equilibrium is stable. 



Uniqueness of Filippov solutions 

A discontinuous dynamical system does not necessarily have a unique Filippov solution starting 
from each initial condition. The situation depicted in Figure El^c) is a qualitative example in which 
multiple Filippov solutions exist for the same initial condition. The following is another example 
of lack of uniqueness. 



Example 11: Vector field with nonunique Filippov solutions 

Consider the vector field X : M ^ R defined by X{x) = sign(a::). For all xq £ M\{0}, the 
system (|10p has a unique Filippov solution starting from xq. However, the system (jlOp has three 
maximal solutions xi,X2,X3 : [0, oo) — > R starting from xq = given by xi{t) = —t, X2{t) = 0, and 
x-iit) =t. U 



We now provide two complementary uniqueness results for Filippov solutions. The first result ISj 
considers the Filippov set-valued map associated with a discontinuous vector field, and identifies 
conditions under which Proposition [S2] in "Caratheodory Solutions of Differential Inclusions" can 
be applied to the resulting differential inclusion. In order to state this result, we need to introduce 
the following definition. The vector field X : M'^ — > R'^ is essentially one-sided Lipschitz onU C R*^ 
if there exists L > such that, for almost all y,y' G U, 

[X{y) - X{y')fiy - y') < L\\y - y'Wl (32) 

The first uniqueness result for Filippov solutions is stated next. 

Proposition 4. Let X : R'^ — > R"' 6e measurable and locally essentially bounded. 
Assume that, for all x G R'^, there exists e > such that X is essentially one-sided Lipschitz on 
B{x,e). Then, for all xq G R*^, there exists a unique Filippov solution of (jlOp with initial condition 
x{0) = Xq. 

Note the parallelism of this result with Proposition [2] for ordinary differential equations with a 
continuous vector field X. Let us apply Proposition U] to an example. 
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Example 12: Vector field with unique Filippov solutions 



Let Q denote the set of rational numbers, and define the vector field X : M — > M by 

Xix) = 




which is discontinuous everywhere on M. Since Q has measure zero in M, the value of the vector 
field at rational points plays no role in the computation of Hence, the associated Filippov 

set-valued map F[X] : R QS(M) is F[X]{x) = {-1}. Since ([32]) holds for ah y,y' Q, there 
exists a unique solution starting from each initial condition, more precisely, x : [0, oo) — > M, where 
x{t) = x{0) -t. ■ 

The hypotheses of Proposition [5] are somewhat restrictive. This assertion is justified by the 
observation that, for d > 1, piecewise continuous vector fields on are not essentially one-sided 
Lipschitz. We justify this assertion in "Uniqueness of Filippov Solutions of Piecewise Continuous 
Vector Fields." Figure IS2I shows an example of a piecewise continuous vector field with unique 
solutions starting from each initial condition. However, this uniqueness cannot be guaranteed by 
means of Proposition 



The following result [18(] identifies sufficient conditions for uniqueness specifically tailored for 



piecewise continuous vector fields. 

Proposition 5. Let X : — > M"' 6e a piecewise continuous vector field, with M'^ = 
Di yjT>2- Let Sx = bndry(Di) = bndry(P2) be the set of points at which X is discontinuous, and 
assume that Sx is a C"^ -manifold. Furthermore, assume that, for i G {1,2}, X^^y. is continuously 
differentiable on T>i and Xy^ — Xy^ is continuously differentiable on Sx- If, for each x G Sx, 
either Xy^{x) points into P2 or Xy^{x) points into Di, then there exists a unique Filippov solution 
of (jlOp starting from each initial condition. 



Note that the continuous differentiability hypothesis on X already guarantees uniqueness of 
solutions on each of the sets Vi and 'D2. Roughly speaking, the additional assumptions on X along 
Sx in Proposition [5] guarantee that uniqueness on is not disrupted by the discontinuities. Under 
the stated assumptions, when reaching Sx, Filippov solutions can cross it or slide along it. The 
situation depicted in Figure El^c) is thus ruled out. 



Examples [2H4] revisited: Unique Filippov solutions for the sliding brick, the nonsmooth 
harmonic oscillator, and the move-away-from-nearest-neighbor interaction law 

As an application of Proposition [5l let us reconsider Example [2j At v = 0, the vector Xy^{0) 
points into T>2, see Figure El^b). Proposition [5] then ensures that there exists a unique Filippov 
solution starting from each initial condition. 

For Example El the vector Xy^ points into P2 at every point in Sx H {x2 > 0}, while the vector 
Xy^ points into Pi at every point in Sx H {x2 < 0}, see Figure [3]^a). Moreover, there is only one 
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solution (the equilibrium solution) starting from (0, 0). Therefore, using Proposition [5l we conclude 
that this system has a unique Filippov solution starting from each initial condition. 



For ExampleHl it is convenient to define T>5 = T>i. Then, at {xi,X2) G bndry('Dj) nbndry(Dj+i)\ 
{(0, 0)}, with i G {1, . . . , 4}, the vector X|^(a;i, X2) points into I'i+i, and the vector X|^-j-j-(xi, X2) 
points into T>i, see Figure [6ja). Moreover, there is only one solution (the equilibrium solution) 
starting from (0,0). Therefore, using Proposition [SJ we conclude that Example H] has a unique 
Filippov solution starting from each initial condition. ■ 

Proposition[5]can also be applied to piecewise continuous vector fields with an arbitrary number of 
partitioning domains, provided that the set where the vector field is discontinuous is a disjoint union 
of surfaces resulting from pairwise intersections of the boundaries of pairs of domains. Alternative 
versions of this result can also be stated for time- varying piecewise continuous vector fields, as well 
as for situations in which more than two domains intersect at a point of discontinuity [3, Theorem 
4 at page 115]. 

The literature contains additional results guaranteeing uniqueness of Filippov solutions tailored 
to specific classes of dynamical systems. For instance, [s^ studies u niq ueness for relay linear 



systems, [ll|] investigates uniqueness for adaptive control systems, while [40| establishes uniqueness 
for a class of discontinuous differential equations whose vector field depends on the solution of a 
scalar conservation law. 



Solutions of control systems with discontinuous inputs 

Let X : M"' X Z// — > M*^, where U C is the set of allowable control- function values, and consider 
the control equation on M'^ given by 

±{t) = X{x{t),u{t)). (33) 

At first sight, a natural way to identify a notion of solution for (j33p is to select a control input, either 
an open-loop u : [0, 00) U, a closed-loop n : M"^ — > Z^, or a combination u : [0, 00) x M.'^ ^ U, 
and then consider the resulting differential equation. When the selected control input n is a 
discontinuous function of x G M*^, then we can consider the solution notions of Caratheodory or 
Filippov. At least two alternatives are considered in the literature. We discuss them next. 



Solutions by means of differential inclusions 

A first alternative to defining a solution notion consists of associating a differential inclusion with 
the control equation (|33p . In this approach, the set-valued map G[X] : R'^ — > !B(M'^) is defined by 

G[X]{x) = {X{x,u) : ueU}. (34) 

In other words, the set G[X](x) captures all of the directions in M"' that can be generated at x with 
controls belonging to U. Consider now the differential inclusion 

x{t) G G[X]{x{t)). (35) 
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A solution of (|33p on [0, ti] C M is defined to be a Caratheodory solution of the differential 
inclusion (j35p . that is, an absolutely continuous map x : [0,ti] R"^ such that ±{t) G G[X]{x{t)) 
for almost all t £ [0, ti]. 

If we choose an open-loop input u : [0, oo) — > W in (jSSp . then a Caratheodory solution of the 
resulting dynamical system is also a Caratheodory solution of the differential inclusion (j35p . Alter- 
natively, it can be shown [l^ that, if X is continuous and U is compact, then the converse is also 
true. The differential inclusion (|35p has the advantage of not focusing attention on a particular 
control input, but rather allows us to comprehensively study and understand the properties of the 
control system. 



Sample- and- hold solutions 

A second alternative to defining a solution notion for the control equation ()33p uses the notion of 
sample-and-hold solution [4l|]. As discussed in the section "Stabilization of control systems," this 



notion plays a key role in the stabilization of asymptotically controllable systems. 

A partition of the interval [to)*i] is an increasing sequence vr = {sj}^Q with sq = tQ and sn = ti. 
The partition need not be finite. The notion of a partition of [to,oo) is defined similarly. The 
diameter of vr is diam(7r) = supjsj — Sj_i : i G {1, . . . , A^}}. Given a control input u : [0,oo) xM'^ — > 
U, an initial condition xq, and a partition vr of [0, ti], a n-solution of (]33p defined on [0, ti] C M is the 
map X : [0, ti] W^, with x{0) = xq, recursively defined by requiring the curve t G Si] i— > x{t), 

for i G {1, . . . , — 1}, to be a Caratheodory solution of the differential equation 

x{t) = u(si_i, (36) 

vr-solutions are also referred to as sample-and-hold solutions because the control is held fixed 
throughout each interval of the partition at the value according to the state at the beginning of the 



interval. Figure [7] shows an example of a TT-solution. The existence of vr-solutions is guaranteed 2J] 
if, for all u gU CI M™-, the map x i— > X{x, u) is continuous. 



Nonsmooth Analysis 



We now consider candidate nonsmooth Lyapunov functions for discontinuous differential equa- 
tions. The level of generality provided by nonsmooth analysis is not always necessary. The stability 
properties of some discontinuous dynamical systems and differential inclusions can be analyzed with 
smooth functions, as the following example shows. 



Examples [5] and [6] revisited: Stability of the origin for the sign function 

We have already established that the vector fields (llip and (I12p in examples [5] and [U respectively, 
have unique Filippov solutions starting from each initial condition. Now, consider the smooth 
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Lyapunov function / : R ^ M, where f{x) = x'^/2. Now, for all x 7^ 0, we have 

V/(x) • X{x) = -\x\ < 0. 

Since, according to (f20l) . = {X{x)} on ]R\{0}, we deduce that the function / is decreasing 

along every Filippov solution of (jlip and (jl2p that starts on ]R\{0}. Therefore, we conclude that 
the equilibrium x = is globally asymptotically stable. ■ 

However, nonsmooth Lyapunov functions may be needed if dealing with discontinuous dynamics, 
as the following example shows. 



Example [3] revisited: The nonsmooth harmonic oscillator does not admit a smooth 
Lyapunov function 



Consider the vector field for the nonsmooth harmonic oscillator in Example O We reason as 



m 



16l |. As stated in the section "Piecewise continuous vector fields and sliding motions," all of 
the Filippov solutions of ^ are periodic, and correspond to the curves plotted in Figure E^b). 
Therefore, if a smooth Lyapunov function / : ^ M exists, then it must be constant on each 
Filippov solution. Since the level sets of / must be one-dimensional, it follows that each curve must 
be a level set. This property contradicts the fact that the function is smooth, since the level sets 
of a smooth function are also smooth, and the Filippov solutions plotted in Figure [3jb) are not 
smooth along the j;2-coordinate axis. ■ 

The above example illustrates the need to consider nonsmooth analysis. Similar examples can 
be found in [H, Section 2.2.2]. It is also worth noting that the presentation in "Stability analysis 
by means of the generalized gradient of a nonsmooth Lyapunov function" boils down to classical 
stability analysis when the candidate Lyapunov function is smooth. 

In this section we discuss two tools from nonsmooth analysis, namely, the generalized gradient 
and the proximal subdifferential [s, 24]. As with the notions of solution of discontinuous differential 



equations, multiple generalized derivative notions are available in the literature when a function 
fails to be differentiable. These notions include, in addition to the two considered in this section, 
the generalized (super or sub) differential, the (upper or lower, right or left) Dini derivative, and 
the contingent derivative 1^, 24, 4^, 43]. Here, we focus on the notions of the generalized gradient 



and proximal subdifferential because of their role in providing stability tools for discontinuous 
differential equations. 



The generalized gradient of a locally Lipschitz function 

Rademacher's theorem 0] states that every locally Lipschitz function is differentiable almost 
everywhere in the sense of Lebesgue measure. When considering a locally Lipschitz function as a 
candidate Lyapunov function, this statement may raise the question of whether to disregard those 
points where the gradient does not exist. Conceivably, the solutions of the dynamical system stay 
for almost all time away from "bad" points where no gradient of the function exists. However, this 
assumption is not always valid. As we show in Example [16] below, the solutions of a dynamical 
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system may insist on staying on the "bad" points forever. In that case, having some gradient-hke 
information is helpful. 

Let / : M'^ — > M be a locally Lipschitz function, and let 17^ C M.'^ denote the set of points where 
/ fails to be differentiable. The generalized gradient df ^ *B(M"') of / is defined by 

= co{ lim V/(xi) : Xj — > x , Xj 5 U il/}, (37) 

woo 

where co denotes convex hull. In this definition, 5" C M*^ is a set of measure zero that can be 
arbitrarily chosen to simplify the computation. The resulting set df{x) is independent of the choice 
of S. From the definition, the generalized gradient of / at x consists of all convex combinations of 
all of the possible limits of the gradient at neighboring points where / is differentiable. Equivalent 
definitions of the generalized gradient are given in 



Some useful properties of the generalized gradient are summarized in the following result [2j]. 



Proposition 6. // / : M'^ — > M is locally Lipschitz at x € M'^, then the following 
statements hold: 

(i) df{x) is nonempty, compact, and convex. 

(a) The set-valued map df : ^ *B(M'^), x df{x), is upper semicontinuous and locally 
bounded at x. 

(Hi) If f is continuously differentiable at x, then df{x) = {V/(x)}. 
Let us compute the generalized gradient for an illustrative example. 



Example 13: Generalized gradient of the absolute value function 

Consider the locally Lipschitz function / : M — > R, where /(x) = |x|, which is continuously 
differentiable everywhere except at 0. Since V/(x) = 1 for x > and V/(x) = —1 for x < 0, 
from ([37|) . we deduce 

5/(0) = co{l,-l} = [-1,1], 

while, by Proposition [Glf^zMj, we deduce df{x) = {1} for x > and df{x) = {—1} for x < 0. ■ 

The next example shows that Proposition [Hf'iiij may not be true if / is not continuously differ- 
entiable. 



Example 14: Generalized gradient of a differentiable but not continuously differen- 
tiable function 

Consider the locally Lipschitz function / : M — > M, where /(x) = x^ sin(^), which is continuously 
differentiable everywhere except at 0, where it is only differentiable. It can be shown that V/(0) = 0, 
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while V/(x) = 2xsin(i) — cos(^) for x ^ 0. Prom (|37p . we deduce that 9/(0) = [—1, 1], which is 
different from {V/(0)} = {0}. This example illustrates that Proposition [6](^mj may not be valid if 
/ is not continuously differ entiable at the point x. ■ 



Computing the generalized gradient 

Computation of the generalized gradient of a locally Lipschitz function is often a difficult task. 
In addition to the brute force approach of working from the definition (|37p . various results are 
available to facilitate this computation. Many results that are valid for ordinary derivatives have a 
counterpart in this setting. We summarize some of these results here, and refer the reader to 
for a complete exposition. In the statements of these results, the notion of a regular function plays 
a prominent role, see "Regular Punctions" for a precise definition. 

Dilation rule. If / : M"^ ^ M is locally Lipschitz at x G M"^ and s G R, then the function sf is 
locally Lipschitz at x, and 

d{sf){x) = sdf{x). (38) 

Sum rule. If /i, /2 : M'^ — > M are locally Lipschitz at x G M"^ and si,S2 G M, then the function 
si/i + S2/2 is locally Lipschitz at x, and 

d{sifi + S2f2){x) C sidfiix) + S2df2{x), (39) 

where the sum of two sets Ai and A2 is defined as Ai + A2 = {ai + 02 : ai £ Ai, a2 £ A2}. 
Moreover, if /i and f2 are regular at x, and si, S2 G [0, 00), then equality holds and si/i + S2/2 is 
regular at x. 

Product rule. If /i, /2 : M'^ ^ M are locally Lipschitz at x G M'^, then the function /1/2 is 
locally Lipschitz at x, and 

5(/l/2)(x) C /2(x)a/i(x) +/i(x)a/2(x). (40) 

Moreover, if /i and /2 are regular at x, and /i(x), f2ix) > 0, then equality holds and /1/2 is regular 
at X. 

Quotient rule. If /i, /2 : M"' — > M are locally Lipschitz at x G M"', and /2(x) / 0, then the 
function /1//2 is locally Lipschitz at x, and 

9(|)(x) C ^(/2(x)9/i(x) - /i(x)9/2(x)). (41) 

Moreover, if /i and — /2 are regular at x, and /i(x) > and f2ix) > 0, then equality holds and 
/1//2 is regular at x. 

Chain rule. If each component of /i : M*^ — > is locally Lipschitz at x G and g : M™ M 
is locally Lipschitz at h{x) G M™', then the function g o h is locally Lipschitz at x, and 

m 

d{goh){x) Oco^^UkCk ■■ {ai,...,am) &dg{h{x)), (Ci, Cm) G 9/^1(2;) x ••• x9/i^(x)|. (42) 

k=l 
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Moreover, if g is regular at each component of h is regular at x, and dg{h{x)) C [0, oo)'", 

then equality holds and g oh \s regular at x. 

The following useful result from fa, Proposition 2.3.12] concerns the generalized gradient of the 
maximum and the minimum of a finite set of functions. 

Proposition 7. For k G {1, • • • jfu}, let f^-.W^^Wbe locally Lipschitz at x ^ W^, 
and define the functions /maxj /min : M'^ ^ M hy 

/max(y) = max{/fc(y) : A; G {1, . . . , m}}, 
/min(y) = min{/fc(y) : G {1, . . . , m}}. 

T/ien, the following statements hold: 
(i) /max o-nd /mill «re locally Lipschitz at x. 

(a) Let /max (3;) denote the set of indices k for which fk{x) = fma.xix)- Then 

9/max(2;) C Co[J{dfi{x) : i G /max(2;)}. (43) 

Furthermore, if fi is regular at x for all i G /max (2;), then equality holds and /max is regular 
at X, 

(Hi) Let /min (3^) denote the set of indices k for which fk{x) = /min (2;). Then 

dfmUx) C co\J{df,{x) : i G Iminix)}. (44) 

Furthermore, if —fi is regular at x for all i G Im\n{x), then equality holds and —/min is regular 
at X. 

It follows from Proposition [7| that the maximum of a finite set of continuously differentiable 
functions is a locally Lipschitz and regular function whose generalized gradient at each point x is 
easily computable as the convex hull of the gradients of the functions that attain the maximum at 

X. 

Example 1131 revisited: Generalized gradient of the absolute value function 

The absolute value function /(x) = |x| can be rewritten as /(x) = max{x,— x}. Both x i— > x 
and X I— > —X are continuously differentiable, and hence locally Lipschitz and regular. Therefore, 
according to Proposition [7](^ij and (ii), f is locally Lipschitz and regular, and its generalized gradient 
is 

r {1}, X > 0, 

a/(x) = |[-l,l], x = 0, (45) 
[{-1}, x<0. 



This result is obtained in Example [13] by direct computation. 
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Example 15: Generalized gradient of the minus absolute value function 

The minimum of a finite set of regular functions is not always regular. A simple example is 
given by g{x) = mm{x, —x} = —\x\, which is not regular at 0, as we show in "Regular Functions." 
However, according to Proposition ^iii), this fact does not mean that its generalized gradient 
cannot be computed. Indeed, 

f{-i}, ^>o, 

dg{x) = x = 0, (46) 

[{!}, a^<0. 

This result can also be obtained by combining (|45p with the application of the dilation rule (|38p to 
the function /(x) = |x| with the parameter s = —1. ■ 

Critical points and directions of descent 

A critical point of / : R'' ^ M is a point x S M'' such that 6 df{x). According to this definition, 
the maximizers and minimizers of a locally Lipschitz function are critical points. As an example, 
X = is the unique minimizer of f{x) = \x\, and, indeed, we see that € df{0) in ()45p . 

If a function / is continuously differentiable, then the gradient V/ provides the direction of 
maximum ascent of / (respectively, —V/ provides the direction of maximum descent of /). When 
we consider a locally Lipschitz function, however, the question of choosing the directions of descent 
among all of the available directions in the generalized gradient arises. Without loss of generality, 
we restrict our discussion to directions of descent, since a direction of descent of — / corresponds 
to a direction of ascent of /, while / is locally Lipschitz if and only if — / is locally Lipschitz. 

Let Ln : «B(M'^) !B(M'^) be the set-valued map that associates to each subset S of M'^ the set 
of least-norm elements of its closure S. If the set S is convex and closed, then the set Ln(S') is a 
singleton, which consists of the orthogonal projection of onto S. For a locally Lipschitz function 
/, consider the generalized gradient vector field Ln{df) ■.M.'^^M.'^ defined by 

X Ln{df){x) = Ln{df{x)). 
It turns out that —hn{df){x) is a direction of descent of / at x G M'^. More precisely 

B, if 

df{x), then there exists T > such that 

fix - tLn(5/)(x)) < fix) - ^11 Ln(a/)(x)||i < /(x), < t < T, (47) 

that is, by taking a small step in the direction — Ln((9/)(x), the function / is guaranteed to decrease 
by an amount that scales linearly with the stepsize. 
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Example 16: Minimum-distance-to-polygonal-boundary function 



Let Q C be a convex polygon. Consider the minimum distance function smg : Q ^ M from a 
point within the polygon to its boundary defined by 

smQ(p) = min{||p - q\\2 : q G bndry(Q)}. 

Note that the value of smg at p is the radius of the largest disk with center p contained in the 
polygon. Moreover, the function smg is locally Lipschitz on Q. To show this, rewrite smg as 

smg(p) = min{dist(p, e) : e is an edge of Q}, 

where dist(p, e) denotes the Euclidean distance from the point p to the edge e. Indeed, the function 
smg is concave on Q. 

Let us consider the generalized gradient vector field corresponding to smg, where the definition 
of smg is extended outside Q by setting smg(p) = — min{||p — q\\2 : q G bndry((5)} for p ^ Q. 
Applying Proposition [7|(^iiij, we deduce that — smg is regular on Q and its generalized gradient at 
p ^ Q is 

9smg(p) = cojue : e edge of Q such that smg(p) = dist(p, e)}, 

where denotes the unit normal to the edge e pointing toward the interior of Q. Therefore, 
at points p in Q for which there exists a unique edge e of Q nearest to p, the function smg is 
differentiable, and its generalized gradient vector field is given by Ln(smg)(p) = Ug. Note that 
this vector field corresponds to the move-away-from-nearest-neighbor interaction law for one agent 
moving in the polygon introduced in Example H) 

At points p of Q where various edges {ei, . . . ,em} are at the same minimum distance to p, the 
function smg is not differentiable, and its generalized gradient vector field is given by the least- 
norm element in cojue^ , ■ ■ ■ , ^e,^}- If P is not a critical point, does not belong to cojuei , • • • , Hg^}, 
and the least-norm element points in the direction of the bisector line between two of the edges in 
{ei, . . . , em}- Figure [9] shows a plot of the generalized gradient vector field of smg on the square 
Q = [—1, 1]^. Note the similarity with the plot in Figure [6|^a) . Indeed, the critical points of smg 



are characterized 4J] by the statement that G Osmg(p) if and only if p belongs to the incenter 
set of Q. 

The incenter set of Q is composed of the centers of the largest-radius disks contained in Q. 
In general, the incenter set is a segment (consider the example of a rectangle). However, if G 
interior (9 smg (p)), then the incenter set of Q is the singleton {p}. ■ 



Nonsmooth gradient flows 

Given a locally Lipschitz function / : M'^ ^ M, the nonsmooth analog of the classical gradient 
descent flow of a differentiable function is defined by 



x{t) = -Ln{df){x{t)). 
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(48) 



According to (|47p. unless the flow is already at a critical point, — Ln{df){x) is a direction of descent 
of / at X. Since this nonsmooth gradient vector field is discontinuous, we have to specify a notion of 
solution for (j48p . In this case, we use the notion of Filippov solution due largely to the remarkable 



fact [361] that the Filippov set-valued map associated with the nonsmooth gradient flow of / given 



by (j48|) is precisely the generalized gradient of the function, as the next result states. 

Proposition 8 (Filippov set- valued map of nonsmooth gradient). 7/ / : M"^ — > M is 
locally Lipschitz, then the Filippov set-valued map F[Ln{df)] : R"^ *B(M'^) of the nonsmooth 
gradient of f is equal to the generalized gradient df ^ *B(M'^) of f, that is, for x G M^, 

F[Ln{df)]ix)=df{x). (49) 



As a consequence of Proposition [8l the discontinuous system (j48p is equivalent to the differential 
inclusion 

x{t) G -df{x{t)). (50) 

Solutions of (148p are sometimes called "slow motions" of the differential inclusion because of the 
use of the least-norm operator, which selects the vector in the generalized gradient of the function 
with the smallest magnitude. How can we analyze the asymptotic behavior of the solutions of this 
system? If the function / is differentiable, then the invariance principle allows us to deduce that, for 
functions with bounded level sets, the solutions of the gradient flow converge to the set of critical 
points. The key tool behind this result is to establish that the function decreases along solutions 
of the system. This behavior is formally expressed through the notion of the Lie derivative. In the 
section "Nonsmooth Stability Analysis" we discuss generalizations of the notion of Lie derivative 
to the nonsmooth case. These notions allow us to study the asymptotic convergence properties of 
the trajectories of nonsmooth gradient flows. 



The proximal subdifferential of a lower semicontinuous function 

A complementary set of nonsmooth analysis tools for dealing with Lyapunov functions arises from 
the concept of proximal subdifferential. This concept has the advantage of being defined for the 
class of lower semicontinuous functions, which is larger than the class of locally Lipschitz functions. 
The generalized gradient provides us with directional descent information, that is, directions along 
which the function decreases. The price we pay for using the proximal subdifferential is that explicit 
descent directions are not generally known to us. Nevertheless, the proximal subdifferential allows 
us to reason about the monotonic properties of the function, which, as we show in the section 
"Stability analysis by means of the proximal subdifferential of a nonsmooth Lyapunov function," 
turn out to be sufficient to provide stability results. The proximal subdifferential is particularly 
appropriate when dealing with convex functions. Here, we briefly touch on the topic of convex 
analysis. For further information, see 



A function / : M'^ ^ M is lower semicontinuous [2J] at x G M*^ if, for all e G (0, oo), there exists 
5 G (0, oo) such that, for y G B{x, 5), f{y) > f{x) — e. The epigraph of / is the set of points lying 
on or above its graph, that is, epi(/) = {{x,\x) G M*^ x M : f{x) < /i} C R^^^ . The function / 
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is lower semicontinuous if and only if its epigraph is closed. The function / : — > R is upper 
semicontinuous at x E M'^ if — / is lower semicontinuous at x. Note that / is continuous at x if and 
only if / is both upper semicontinuous and lower semicontinuous at x. 

For a lower semicontinuous function / : M'^ — > M, the vector ( S R*^ is a proximal subgradient of 
f X G R'^ if there exist a, 5 G (0, cxd) such that, for all y E B(x, 6), 

f{y)>f{x)+C{y-x)-a^\\y-x\\l (51) 

The set dpf{x) of all proximal subgradients of / at x is the proximal suhdifferential of f at x. The 
proximal subdifferential at x, which may be empty, is convex but not necessarily open, closed, or 
bounded. 

Geometrically, the definition of a proximal subgradient can be interpreted as follows. Equa- 
tion ()5ip is equivalent to saying that, in a neighborhood of x, the function y i— > f{y) majorizes the 
quadratic function y i— > f{x) -|- (^{y — x) — (T^||y — x\\2. In other words, there exists a parabola that 
locally fits under the epigraph of / at (x, /(x)). This geometric interpretation is useful for explicitly 
computing the proximal subdifferential, see Figure [TOlf a) for an illustration. An additional geomet- 
ric characterization of the proximal subdifferential of / can be given in terms of normal cones and 
the epigraph of /. Given a closed set S C R*^ and x € S*, the proximal normal cone Ns{x) to 5* is 
the set of all y G R'^ such that x is the nearest point in S" to x -|- \y, for A > sufficiently small. 
Then, £ dpf{x) if and only if 

(C-l) GiVepi(/)(x,/(x)). (52) 
Figure [TOlfb) illustrates this geometric interpretation. 



Example 17: Proximal subdifFerentials of the absolute value function and its negative 



Consider the locally Lipschitz functions /, 5 : R - 
geometric interpretation of (fSTIl . it can be seen that 



I, /(x) = |x| and g{x) 



Using the 



dpfix) 



dpg{x) 





X < 0, 


< [-1,1] 


, X = 0, 


U-1}, 


X > 0, 


fi-l}, 


X < 0, 


< 0, 


X = 0, 


.{1}, 


X > 0. 



(53) 



(54) 



Note that ([53|) - ([5l|) is an example of the fact that dp{—f) and —dpf are not necessarily equal. 
The generalized gradient of g in (j46p is different from the proximal subdifferential of g in (154p . ■ 

Unlike the case of the generalized gradient, the proximal subdifferential may not coincide with 
V/(x) when / is continuously differentiable. The function / : R ^ R, where /(x) = — |xp/^, is 
continuously differentiable with V/(0) = 0, but 9p/(0) = 0. In fact, a continuously differentiable 
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function on M with an empty pro ximal subdifferential almost everywhere is provided in [43] ■ How- 
ever, the density theorem (cf. [2J, Theorem 3.1]) states that the proximal subdifferential of a lower 
semicontinuous function is nonempty on a dense set of its domain of definition, although a dense 
set can have zero Lebesgue measure. 

On the other hand, the proximal subdifferential can be more useful than the generalized gradient 
in some situations, as the following example shows. 



Example 18: Proximal subdifferential of the square root of the absolute value 



Consider the function / : M — > M, where f{x) = y which is continuous at 0, but not locally 
Lipschitz at 0, which is the global minimizer of /. The generalized gradient does not exist at 0, 
and hence we cannot characterize this point as a critical point. However, the function / is lower 
semicontinuous with proximal subdifferential. 



dpfix) 



{27;}' ^>0' 

M, x = 0, (55) 

As we discuss later, it follows from (|55p that is the unique global minimizer of /. ■ 

If / : M'^ — > R is locally Lipschitz at x G M'^, then the proximal subdifferential of / at x is bounded. 
In general, the relationship between the generalized gradient and the proximal subdifferential of a 
function / that is locally Lipschitz at x G M*^ is expressed by 

df{x) = co{ lim C„ G M'^ : Cn G dpf{xn) and lim Xn = x}. 



n—foo 



Computing the proximal subdifferential 

As with the generalized gradient, computation of the proximal subdifferential of a lower semicon- 
tinuous function is often far from straightforward. Here we provide some useful results following 



the exposition in j2J]. 



Dilation rule. If / : ^ M is lower semicontinuous at 2; G and s G (0,oo), then the 
function sf is lower semicontinuous at x, and 

dp{sf){x) = s dpfix). (56) 

Sum rule. If /i, /2 : R'^ ^ R are lower semicontinuous at x G M.'^, then the function fi + /2 is 
lower semicontinuous at x, and 

dpfiix) + dpf2{x) C dp{h + /2)(x). (57) 

Moreover, if either fi or /2 are twice continuously differentiable, then equality holds. 
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Fuzzy sum rule. If /i, /2 : ^ M is lower semicontinuous at x G M^, C, G dp[fi + 72) (x), and 
e > 0, then there exist xi,X2 G B{x,e) with — fi{x)\ < e, i £ {1,2}, such that 

C G dpfiixi) + ap/2(x2) + eB{0, 1). (58) 

Chain rule. Assume that either /i : R'^ — > hnear and g : — > M lower semicontinuous 
at h{x) G M™, ov h -.W^ locally Lipschitz at x G M'^ and 5 : M"" ^ M locally Lipschitz at 

h{x) G R™. If C e dp{g o /i)(x) and e > 0, then there exist x G R*^, y G R"", and 7 G dpg{y) with 
max{||x — x||2, \\y — h{x)\\2} < e such that \\h{x) — h{x)\\2 < e and 

CG ap(7^/i)(x) +65(0,1), (59) 

where j'^h : ^ R is defined by (7-^/1) (x) = 7'^/i(x). 

The statement of the chain rule (|59p shows a characteristic feature of the proximal subdifferential. 
That is, rather than at an specific point of interest, the proximal subdifferential can only be 
expressed with objects evaluated at neighboring points. 

Computation of the proximal subdifferential of a twice continuously differentiable function is 
particularly simple. If / : R*^ — > R is twice continuously differentiable on the open set U C R'^, 
then, for all x £ U, 

dpfix) = {V/(x)}. (60) 

This simplicity also works for continuously differentiable convex functions, as the following re- 
sult [20I . 24 . 47 1 states. Note that every convex function on R"' is locally Lipschitz, and hence 



continuous. 

Proposition 9. // / : R*^ ^ R is convex, then the following statements hold: 
(i) For X G R"*, C G dpf{x) if and only if for all y G R'^, f{y) > f{x) + C{y - x). 



(a) The set-valued map dpf : R — > *B(R ), x dpf{x), takes nonempty, compact, and convex 
values, and is upper semicontinuous and locally bounded. 

(Hi) If, in addition, f is continuously differentiable, then, for all x G R'', dpf{x) = {V/(x)}. 



Example 1171 revisited: Proximal subdifferential of the absolute value function 

The computation (j53p of the proximal subdifferential of the function /(x) = |x| can be performed 
using Proposition [9] by observing that / is convex. By Proposition [9](^iJ, for x = 0, C ^ dpf{0) if 
and only if, for all y G R, 

\y\>\o\ + C{y-o) = Cy. (6i) 

Hence, for y > 0, ([6T]) implies C ^ (~oo, 1], while, for y < 0, ([6T]) implies that C G [— l,+oo). 
Therefore, ( G (— 00, 1] n [— l,+oo), and dpf{0) = [—1,1]. By Proposition [9](^mJ, for x > 0, 
dpf{x) = {1}, and, for x < 0, dpf{x) = {-1}. ■ 
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Regarding critical points, if 2; is a local minimizer of the lower semicontinuous function / : M'^ ^ 
M, then G dpf{x). Conversely, if / is convex, and E dpf{x), then x is a global minimizer of 
/. For the study of maximizers, instead of lower semicontinuous function, convex function, and 
proximal subdifferential, the relevant notions are upper semicontinuous function, concave function, 
and proximal superdifferential, respectively 24l ]. 



Gradient differential inclusion of a convex function 

It is not always possible to associate a nonsmooth gradient flow with a lower semicontinuous 
function because the proximal subdifferential might be empty almost everywhere. However, follow- 
ing Proposition [Sl^iij, we can associate a nonsmooth gradient flow with each convex function, as we 
briefly discuss next [49|. 

Let / : M'^ ^ M be convex, and consider the gradient differential inclusion 

x{t) E -dpf{x{t)). (62) 

Using the properties of the proximal subdifferential given by Proposition [9](^mJ, the existence of a 
Caratheodory solution of ([62]) starting from every initial condition is guaranteed by Proposition [Sll 
Moreover, the uniqueness of Caratheodory solutions can be established as follows. Let x, y E M'^, 
and take Ci E —dpf{x) and C2 E —dpf{y). Using Proposition [9](^ij, we have 

f{y) > fix) - Ci(y - x), fix) > fiy) - Ux - y). 

From here, we deduce —Ciiv — x) < fiy) — fix) < — C2(y — x), and therefore (C2 — Ci)(y — x) < 0, 
which, in particular, implies that the set-valued map x 1-^ —dp fix) satisfies the one-sided Lipschitz 
condition ()S2p . Proposition IS2I then guarantees that there exists a unique Caratheodory solution 
of (I62p starting from every initial condition. 



Nonsmooth Stability Analysis 

In this section, we study the stability of discontinuous dynamical systems. Unless explicitly 
mentioned otherwise, the stability notions employed here correspond to the usual ones for differ- 



ential equations, see, for instance 50(]. The presentation focuses on the time-invariant differential 
inclusion 

xit) E J'ixit)), (63) 

where : M."^ ^ *B(M'^). Throughout the section, we assume that the set-valued map satisfies 
the hypotheses of Proposition ISH so that the existence of solutions of (I63p is guaranteed. The 
scenario (j63p has a direct application to discontinuous differential equations and control systems. 
For instance, the results presented here apply to Caratheodory solutions by taking a singleton- 
valued map, to Filippov solutions by taking = F[X], and to control systems by taking = G[X]. 



Before proceeding with our exposition, we recall that solutions of a discontinuous system are not 
necessarily unique. Therefore, when considering a property such as Lyapunov stability, we must 
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specify whether attention is being paid to a particular solution starting from an initial condition 
("weak") or to all the solutions starting from an initial condition ("strong"). As an example, the 
set M C is weakly invariant for (j63p if, for each xq G M, M contains at least one maximal 
solution of (j63p with initial condition xq. Similarly, M C M*^ is strongly invariant for (j63p if, 
for each xq E M, M contains every maximal solution of (j63p with initial condition xq- Rather 
than reintroducing weak and strong versions of standard stability notions, we rely on the guidance 
provided above and the context for the specific meaning in each case. Detailed definitions are given 
in 



141,1181. 



The following discussion requires the notion of a limit point of a solution of a differential inclusion. 
The point G M*^ is a limit point of a solution t ^ x{t) of ([63P if there exists a sequence {tnjnGN 
such that x{tn) x^, as n — > oo. We denote by Q{x) the set of limit points of t ^ x{t). Under the 
hypotheses of Proposition ISll Q{x) is a weakly invariant set. Moreover, if the solution t i— > x{t) 
lies in a bounded, open, and connected set, then Q{x) is nonempty, bounded, connected, and 



x{t) — > Sl(x) as t ^ oo, see jl8l |. 



Stability analysis by means of the generalized gradient of a nonsmooth Lyapunov 
function 

In this section, we discuss nonsmooth stability analysis results that use locally Lipschitz functions 
and generalized gradients. We present results taken from various sources in the literature. This 
discussion is not intended to be a comprehensive account of such a vast topic, but rather serves as 
a motivation for further exploration. The books , [3] and journal articles [sil, [s^ . [sil . [s^ ] provide 
additional information. 

Lie derivative and monotonicity 

A common theme in stability analysis is establishing the monotonic evolution of a candidate 
Lyapunov function along the trajectories of the system. Mathematically, the evolution of a function 
along trajectories is captured by the notion of Lie derivative. Our first task is to generalize this 



notion to the setting of differential inclusions following [53] , see also [5l|, 152] . 



Given a locally Lipschitz function / : M"^ ^ M and a set-valued map J- : 03 (M*^), the 

set-valued Lie derivative Cj^f : *B(M) of / with respect to .7^ at x is defined as 

Cjrf[x) = {o G M : there exists v G !F{x) such that ("^v = a for all ( G df{x)}. (64) 

If J- takes convex and compact values, then, for each x G M'^, Cj^f{x) is a closed and bounded 
interval in M, possibly empty. If / is continuously differentiable at x, then 

Crf{x) = {{Vf{x)fv : V G .^(x)}. 

The usefulness of the set-valued Lie derivative stems from the fact that it allows us to study how 
the function / evolves along the solutions of a differential inclusion without having to explicitly 
obtain the solutions. Specifically, we have the following result, see [s^ ]. 
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Proposition 10. Let x : [0,ti] ^ be a solution of the differential inclusion ()63p . 
and let f '.W^ ^ be locally Lipschitz and regular. Then, the following statements hold: 

(i) The composition t f[x[t)) is differentiable at almost all t G [0,ti]. 
(ii) The derivative of t f{x{t)) satisfies 

G Crf{x{t)) for almost every t £ [0,ti]. (65) 
A similar result can be established for a larger class of functions when the set- valued map J-' is 



singleton- valued [5J, l55[ . 



Given the discontinuous vector field X : M'^ — > M"', consider the Filippov solutions of (jlOp . In this 
case, with a slight abuse of notation, we denote Cxf = ^F[x]f- Note that if X is continuous at 
X, then F[X]{x) = {X{x)}, and therefore, Cxf{x) corresponds to the singleton {Cxf{x)}, whose 
sole element is the usual Lie derivative of / in the direction of X at x. 



Example [3] revisited: Monotonicity in the nonsmooth harmonic oscillator 

For the nonsmooth harmonic oscillator in Example [3l consider the locally Lipschitz and regular 

2 

map / : — > M, f{xi,X2) = \xi\ + Recall that Figure [3)[b) shows the contour plot of /. Let 
us determine how / evolves along the Filippov solutions of the dynamical system by looking at the 
set-valued Lie derivative. First, we compute the generalized gradient of /. To do so, we rewrite / 
as f{xi,X2) = maxl^i, —xi} + and apply Proposition [Tlf'iij and the sum rule to find 



df{xi,X2) 



{(sign(xi),X2)}, xi / 0, 
[-1, 1] X {X2}, xi = 0. 



With this information, we can compute the set-valued Lie derivative Cxf ■ IR^ — > 53 (M) as 

Cxf {xi,X2) -- 



{0}, XI /O, 

0, xi = and X2 / 0, (66) 
_{0}, xi = and X2 = 0. 



From (|65|) and (j66|) we conclude that / is constant along the Filippov solutions of the discontinuous 
dynamical system. Indeed, the level sets of / are exactly the curves described by the solutions of 
the system in Figure EH^b). ■ 



Stability results 

The above discussion on monotonicity is the stepping stone to stability results using locally 
Lipschitz functions and generalized gradient information. Proposition [10] provides a criterion for 
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determining the monotonic behavior of a locally Lipschitz function along the solutions of discon- 
tinuous dynamics. This result, together with the appropriate positive definite assumptions on the 
candidate Lyapunov function, allows us to synthesize checkable stability tests. We start by formu- 
lating [s^ l the natural extension of Lyapunov's stability theorem for ordinary differential equations. 
Recall that, at each x E M'^, the Lie derivative Cjrf[x) is a set contained in M. For the empty set, 
we adopt the convention max0 = — oo. 

Theorem 1. Let T : ^ !B(M'^) he a set-valued map satisfying the hypotheses of 
Proposition \S1\ let Xe be an equilibrium of the differential inclusion ([63]), and let T? Q be an 
open and connected set with Xe € T). Furthermore, let f : ^ W be such that the following 
conditions hold: 

(i) f is locally Lipschitz and regular on D. 
(a) f{xe) = 0, and f{x) > for x G 'D\{xe}. 
(Hi) max£jF/(x) < for each x £T>. 

Then, Xe is a strongly stable equilibrium of ([63]) . In addition, if (Hi) above is replaced by 
(Hi)' max£jF/(x) < for each x G D\{xe}, 
then Xe is a strongly asymptotically stable equilibrium of (j63p . 
Let us apply this result to the nonsmooth harmonic oscillator. 



Example [3] revisited: Stability analysis of the nonsmooth harmonic oscillator 

2 

The function (xi,X2) \xi\ + ^ satisfies hypotheses (i)-(iii) of Theorem [T] on 2? = W^. Therefore, 
we conclude that is a strongly stable equilibrium. The phase portrait in Figure [3|^a) indicates 
that is not strongly asymptotically stable. Using Theorem [H it can be shown that the nonsmooth 
harmonic oscillator under dissipation, with vector field (xi,X2) i— > (x2, — sign(xi) — /csign(x2)), 
where k > 0, has as a strongly asymptotically stable equilibrium. The phase portrait of this 
system is plotted in Figure [Tlla) . while several Filippov solutions are plotted in Figure fTTTb). ■ 



Another useful result in the theory of differential equations is the invariance principle [50|. In 
many situations, this principle allows us to determine the asymptotic convergence properties of the 
solutions of a differential equation. Here, we build on the above discussion to present a general- 
ization to differential inclusions (|63p and nonsmooth Lyapunov functions. This principle is thus 
applicable to discontinuous differential equations. The formulation is taken from [5^, which slightly 



generalizes the presentation in [51]. 



Theorem 2. Let J=' : R'^ ^ «B(M'^) be a set-valued map satisfying the hypotheses of 
Proposition lSl[ and let f : M.'^ ^ R be a locally Lipschitz and regular function. Let S C M."^ be 
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compact and strongly invariant for (j63p . and assume that max Cj^f{y) < for each y £ S. Then, 
all solutions x : [0, cxo) — > M"' of (j63p starting at S converge to the largest weakly invariant set M 
contained in 

Moreover, if the set M consists of a finite number of points, then the limit of each solution starting 
in S exists and is an element of M . 

We now apply Theorem [2] to nonsmooth gradient flows. 



Stability of nonsmooth gradient flows 



Consider the nonsmooth gradient flow (j48p of a locally Lipschitz function / : M — > R. Assmne 
further that / is regular. Let us examine how / evolves along the solutions of the flow using 
the set-valued Lie derivative. Given x G M*^, let a G ^~Ln(df)f{^)- By definition, there exists 
V G F[—Ln{df)]{x) such that 

a = C^v for all C G df{x). (67) 

Recall from ()49p that F[— Ln((9/)](x) = —df{x). Since (j67p holds for all elements in the generalized 
gradient of / at rc, we can choose in particular Q = —v G df{x). Therefore, 

a = {-v^v = -\\v\\l<{). (68) 

Prom ([68]) . we conclude that all of the elements of C^i^n(df)f belong to (— oo,0], and therefore, 
from (j65p . / monotonically decreases along the solutions of its nonsmooth gradient flow (j48p . 
Moreover, we deduce that G ^-Ln(df)f{x) if only if G df{x), that is, if x is a critical point 
of /. The application of the Lyapunov stability theorem and the invariance principle in theorems [1] 
and El respectively, gives rise to the following nonsmooth counterpart of the classical smooth result 



for a gradient flow 56|] 



Proposition 11. Let f : ^ W be locally Lipschitz and regular. Then, the strict 
minimizers of f are strongly stable equilibria of the nonsmooth gradient flow (|48p of f . Furthermore, 
if the level sets of f are bounded, then the solutions of the nonsmooth gradient flow asymptotically 
converge to the set of critical points of f . 



The following example illustrates the above discussion. 



Example 1161 revisited: Stability analysis of the nonsmooth gradient fiow^ of — smg 

Consider the nonsmooth gradient flow of — smQ, which is the minimum-distance-to-polygonal- 
boundary function introduced in Example 1 161 Proposition [5] guarantees Uniqueness of solutions for 
this flow. Regarding convergence, Proposition [11] guarantees that solutions converge asymptotically 



37 



to the incenter set. Indeed, the incenter set is attained in finite time, and hence each solution 



converges to a point of the incenter set [4J]. The nonsmooth gradient flow can be interpreted 
as a sphere-packing algorithm in the sense that, starting from an arbitrary initial point, the flow 
monotonically maximizes the radius of the largest disk contained in the polygon (that is, smg!) 
until it reaches an incenter point. This fact is illustrated in Figure [H[ 

What if, instead, we want to pack an arbitrary number n of spheres within the polygon? It turns 
out that the move-away-from-nearest-neighbor interaction law is a discontinuous dynamical system 
that solves this problem, where the solutions are understood in the Filippov sense. Figure [T3l 
illustrates the evolution of this dynamical system. The stability properties of this law can be 
determined using the Lyapunov function Tisp ■ — > M defined by 

Wsp(pi, . . . ,Pn.) = min{i||pi -pj||2,dist(pi,e) : i ^ j £ {I, . . . ,n}, e edge of Q}. (69) 

The value of Tisp corresponds to the largest radius such that spheres with centers pi, . . . ,pn and 
radius TCspipi, ■ ■ ■ ,Pn) ht within the environment and do not intersect each other (except at most 
at the boundary). The function Tisp is locally Lipschitz, but not concave, on Q, and its evolution is 
monotonically nondecreasing along the move-away-from-nearest-neighbor dynamical system. The 



reader is referred to [4J] for details on the relationship between Tisp and the minimum distance 
function smg, as well as additional discontinuous dynamical systems that solve this sphere-packing 
problem and other exciting geometric optimization problems. ■ 



Finite-time-convergent nonsmooth gradient flows 

Convergence of the solutions of a dynamical system in finite time is a desirable property in various 
settings. For instance, a motion planning algorithm is effective if it can steer a robot from the initial 
to the final configuration in finite time rather than asymptotically. Another example is given by a 
robotic network trying to agree on the exact value of a quantity sensed by each agent. Reaching 
agreement in finite time allows the network to use precise information in the completion of other 
tasks. 

Broadly applicable results on finite-time convergence for discontinuous dynamical systems can 



be found in [36l . l57l . l58l . l59l |. Here, we briefly discuss the finite-convergence properties of a class of 
nonsmooth gradient flows. Let / : M"^ — > M be a continuously difFerentiable function and assume 
that all of the level sets of / are bounded. It is well known [5g] that all solutions of the gradient 
flow x{t) = —S/f{x{t)) converge asymptotically toward the set of critical points of /, although 
the convergence does not occur in finite time. Here, we slightly modify the gradient flow into 
two different nonsmooth flows that achieve finite-time convergence. Consider the discontinuous 
differential equations 

x(t) = _^ZM!)L (70) 

l|V/(x(i))||2' ^'"^ 

x{t) = -sign{Vf{x{t))), (71) 

where || • ||2 denotes the Euclidean distance, and sign(a;) = (sign(a;i), . . . , sign(a;(i)) G W^. The 
nonsmooth vector field (j70p always points in the direction of the gradient with unit speed. Alter- 
natively, the nonsmooth vector field (j7ip specifies the direction of motion by means of a binary 
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quantization of the direction of the gradient. The following result 57[ characterizes the properties 
of the Filippov solutions of (fTOj) and (fTT]) . 



Proposition 12. Let f : Mf^ — > M 6e a twice continuously dijferentiable function. Let 
S C M*^ be compact and strongly invariant for (j70p (resp., for (j7ip ). If the Hessian of f is positive 
definite at every critical point of f in S, then every Filippov solution of (I70p (resp. (j7ip ) starting 
from S converges in finite time to a minimizer of f. 



The proof of Proposition [12] builds on the stability tools presented in this section. Specifically, 
the invariance principle can be used to establish convergence toward the set of critical points of 
/. Finite-time convergence can be established by deriving bounds on the evolution of / along 
the solutions of the discontinuous dynamics using the set-valued Lie derivative. This analysis also 
allows us to provide upper bounds on the convergence time. A more comprehensive exposition of 
results that guarantee finite-time convergence of general discontinuous dynamics is given in 57 1 . 



Example 19: Finite-time consensus 

The ability to reach consensus, or agreement, upon some (a priori unknown) quantity is critical 
for multi-agent systems. Network coordination problems require that individual agents agree on 
the identity of a leader, jointly synchronize their operation, decide which specific pattern to form, 
balance the computational load, or fuse consistently the information gathered on some spatial 
process. Here, we briefly comment on two discontinuous algorithms that achieve consensus in finite 



time, following [57 1 



Consider a network of n agents with states pi,...,pn G M. Let G = {{1, . . . ,n}, E) be an 
undirected graph with n vertices, describing the topology of the network. Two agents i and j 
agree if and only if pi = pj. The disagreement function ■ 1^" ~^ [0, oo) quantifies the group 
disagreement 



2 ^ P^''- 



It is known [60|] that, if the graph is connected, the gradient flow of achieves consensus with an 
exponential rate of convergence. Actually, agents agree on the average value of their initial states 
(this consensus is called average consensus). If G is connected, the nonsmooth gradient flow ([7n|) of 
$G achieves average consensus in finite time, and the nonsmooth gradient flow (j7ip of ^g achieves 
consensus on the average of the maximum and the minimum of the initial states in finite time, 
see |57|]. ■ 



Stability analysis by means of the proximal subdifferential of a nonsmooth Lya- 
punov function 

This section presents stability tools for differential inclusions using a lower semicontinuous func- 
tion as a candidate Lyapunov function. We use the proximal subdifferential to study the monotonic 

39 



evolution of the candidate Lyapunov function along the solutions of the differential inclusion. As 
in the previous section, we present a few representative and useful results. We refer the interested 
reader to [2J, |25[ for a more detailed exposition. 



Lie derivative and monotonicity 

Let P C R'^ be an open and connected set. A lower semicontinuous function / : R'^ ^ M is weakly 
nonincreasing on V for a set-valued map .7^ : M'^ — > *B(M'^) if, for all y S P, there exists a solution 
X : [0, ti] of the differential inclusion (j63p starting at y and contained in 2? that satisfies 

f{x{t)) < /(x(0)) = f{y) for alH G [0,h]. 

If, in addition, / is continuous, then being weakly nonincreasing is equivalent to the property of 
having a solution starting at y such that 1 1— > is monotonically nonincreasing on [0,ti]. 

Similarly, a lower semicontinuous function / : M*^ ^ M is strongly nonincreasing on T> for a 
set- valued map F -.W^ ^ !B(M'^) if, for all y G 2?, all solutions x : [0, ti] — > M"^ of the differential 
inclusion ()63p starting at y and contained in T) satisfy 

f{x{t)) < /(x(0)) = f{y) for all t G [0, h]. 

Note that / is strongly nonincreasing if and only if t f[x{t)) is monotonically nonincreasing on 
[0, ti] for all solutions 1 1— > x{t) of the differential inclusion. 

Given the set-valued map T ^ *B(M'^), which takes nonempty, compact values, and the lower 
semicontinuous function / : M'^ ^ M, the lower and upper set-valued Lie derivatives Cjrf, Cj^f : 
— > *B(M) of / with respect to at y are defined by, respectively, 

Cjrfiy) = {a G M : there exists C G dpf{y) such that a = mm{('^v : v G J^{y)}}, 
^J^f{y) — {a G M : there exists C, G dpf{y) such that a = max{('^v : v G J^{y)}}. 

If, in addition, takes convex values, then for each ( G dpf{y), the set {C"^^^ : v G ^{y)} is a 
closed interval of the form [min{(^"'"u : v G J-{y)},m.ax{(^'^v : v G J-{y)}]. Note that the lower 
and upper set-valued Lie derivatives at a point y might be empty. 

The lower and upper set- valued Lie derivatives play a role for a lower semicontinuous function that 
is similar to the role played by the set-valued Lie derivative Cj^f for a locally Lipschitz function. 
These objects allow us to study how / evolves along the solutions of a differential inclusion without 



having to obtain the solutions in closed form. Specifically, we have the following result 2j] . In this 
and in forthcoming statements, it is convenient to adopt the convention sup0 = —oo. 



Proposition 13. Let : *B(IR ) be a set-valued map satisfying the hypotheses of 

Proposition \S1[ and consider the associated differential inclusion (|63p . Let / : M'^ ^ M 6e a lower 
semicontinuous function, and let V C.W^ be open. Then, the following statements hold: 

(i) The function f is weakly nonincreasing on T> if and only if 

snp^j^fiy) < 0, for all y eV. 
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(ii) If, in addition, either T is locally Lipschitz on T>, or T is continuous on D and f is locally 
Lipschitz on D, then f is strongly nonincreasing on D if and only if 



sup Cyrf{y) < 0, for all y eV. 
Let us illustrate this result in a particular example. 

Example 20: Cart on a circle 



Consider, following 25|, l6]|; the control system on 



xi = {xf — x\)u, (72) 
X2 = 2xiX2U, (73) 

with M G M. Equations (I72p - (l73p are named after the fact that the integral curves of the vector 
field (xi,X2) ^ (xf — x^, 2x1X2) are circles with center on the X2-axis and tangent to the xi-axis, 
see the phase portrait in Figure fT^a). 

Let X : xM ^ m2 defined by X{{xi,X2),u) = {xj-xj, 2x1X2) u. Following (j34p , consider the 
associated set-valued map G[X] : ^(M?) defined by G[X](xi,X2) = {X{(xi,X2),u) : u£R}. 

Since G[X] does not take compact values, we instead take a nondecreasing map a : [0, 00) [0, 00), 
and consider the set- valued map J-^ : M? — > *B(M^) given by 

:F^{xi,X2) = {X{{xi,X2),u) eR^ : \u\ < a{\\{xi,X2)\\2)}. (74) 

Consider the locally Lipschitz function / : — > M defined by 

Ei±£2 X 7^ 

0, X = 0. 

The level set curves of / are depicted in Figure [T^c). Let us determine how / evolves along the 
solutions of the control system by using the lower and upper set-valued Lie derivatives. First, we 
compute the proximal subdifferential of /. Using the fact that / is twice continuously differentiable 
on the open right and left half-planes, together with the geometric interpretation of proximal 
subgradients, we obtain 



dpf {xi,X2) 



{( 



xl+xl-2xiy/xl+x^ X2{2xi+^xf+x^) 
xf+x^+xi^xf+xl ' (xi+^xf+x^)^ 

' x\+x\+2xi^ x\^x\ X2{-2xi+^xl+x^) 



Xi > 0, 

xi = 0, 



« xl+x'^+2xiy/xl+xl X2{-2xi+^xl+x^) \\ X <0 

x\+x\-x\^J x\+x\ ' (xi--y/aif+x|)2 //' 
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With this information, we compute the set 

{C'^v : C ^ dpf{xi,X2), V £ Ta{xi,X2)} 

{u. 

0, 
{- 



u 



x\+x1-x-i^ x'l+ 



■ \u\ < Cr{\\{xi,X2)\\2)}, Xi > 0, 

xi = 0, 

=f : \u\ < (7{\\{xi,X2)\\2)}, a;i < 0. 



We are now ready to compute the lower and upper set-valued Lie derivatives as 



£^J{XI,X2) 



(^tJ{xx,X2) 



-<y\\\\X\,X2)\\2] 



^x\^x\ + \x^\ 



-oo, 



, a:i 7^ 0, 
x\ = 0, 

-oo, x\ = 0. 



(75) 



(76) 



Therefore, sup ^J{xi,X2) < for all (xi,X2) G M^. Now using Proposition I13l(^i). we deduce that 
/ is weakly nonincreasing on M^. Since / is continuous, this fact is equivalent to saying that there 
exists a control input u such that the solution t t-^ x{t) of the dynamical system resulting from 
substituting u in (j72p - (j73p has the property that 1 1— > f{x{t)) is monotonically nonincreasing. ■ 



Stability results 

The results presented in the previous section establishing the monotonic behavior of a lower semi- 
continuous function allow us to provide tools for stability analysis. We present here an exposition 
parallel to the discussion for the locally Lipschitz function and generalized gradient case. We start 
by presenting a result on Lyapunov stability that follows from the exposition in [24 1. 



Theorem 3. Let : ^ *B(M ) be a set-valued map satisfying the hypotheses of 
Proposition \S1[ Let x^ be an equilibrium of the differential inclusion (j63p , and let P C R'^ 6e a 
domain with Xe £ V. Let / : R*^ — > R and assume that the following conditions hold: 



(i) T is continuous on T> and f is locally Lipschitz on T>, or T is locally Lipschitz on T) and f 
is lower semicontinuous on D, and f is continuous at x^. 

(a) f{xe) = 0, and f{x) > for x G D\{xe}. 

(Hi) sup Cjrf{x) < for all x £ V. 

Then, x^ is a strongly stable equilibrium of (|63p . In addition, if (Hi) above is replaced by 



(Hi)' sup£jF/(x) < for all x G 'D\{xe}, 



then Xg is a strongly asymptotically stable equilibrium of ([63]) . 
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A similar result can be stated for weakly stable equilibria substituting (i) hj ^^(i)' / is continuous 
on T>" and the upper set- valued Lie derivative by the lower set-valued Lie derivative in (in) and 
(in)'. Note that, if the differential inclusion (j63p has a unique solution starting from every initial 
conditions, then the notions of strong and weak stability coincide, and it is sufficient to satisfy the 
simpler requirements (i)' and (Hi)' for weak stability. 

Similarly to the case of a continuous differential equation, global asymptotic stability can be 
established by requiring the Lyapunov function / to be continuous and radially unbounded. The 
equivalence between global strong as ymp totic stability and the existence of infinitely differentiable 



Lyapunov functions is discussed in 6^. This type of global result is commonly invoked when 



dealing with the stabilization of control systems by referring to control Lyapunov functions [25[ or 
Lyapunov pairs 2J]. Two lower semicontinuous functions /, 5 : M"* ^ R are a Lyapunov pair for 



an equilibrium Xe S M'^ if they satisfy f{x),g{x) > for all x G M'^, and g{x) = if and only if 
X = Xe', / is radially unbounded, and, moreover, 

sup£^/(x) < -g{x) for ah x e R"^. 

If an equilibrium Xg of (j63p admits a Lyapunov pair, then there exists at least one solution starting 



from every initial condition that asymptotically converges to the equilibrium, see 24l ]. 



Example 1201 revisited: Asymptotic stability of the origin in the cart example 

As an application of the above discussion and the version of Theorem [3] for weak stability, consider 
the cart on a circle introduced in Example I20[ Setting x^, = (0,0) and V = M?, and taking into 
account the computation ([75]) of the lower set-valued Lie derivative, we conclude that (0, 0) is a 
globally weakly asymptotically stable equilibrium. ■ 

We now turn our attention to the extension of the invariance principle for differential inclusions 
using the proximal subdifferential of a lower semicontinuous function. The following result can be 



derived from the exposition in 24i |. 



Theorem 4. Let : M. ^ !B(M ) be a set-valued map satisfying the hypotheses of 
Proposition \S1[ and let f : M. Assume that either T is continuous and f is locally Lipschitz, 

or J- is locally Lipschitz and f is continuous. Let S C M*^ be compact and strongly invariant 
for (I63p . and assume that sup Cj^f{y) < for all y £ S. Then, every solution x : [0, oo) — > M"^ 
of (|63p starting at S converges to the largest weakly invariant set M contained in 



Sn{yGRd . OG £^/(y)}. 



Moreover, if the set M consists of a finite number of points, then the limit of each solution starting 
in S exists and is an element of M . 

Next, we apply Theorem |4] to gradient differential inclusions. 



Stability of gradient differential inclusions for convex functions 
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Consider the gradient differential inclusion (|62p associated with the convex function / : M'^ ^ M. 
We study here the asymptotic behavior of the solutions of (|62p . From the section "Gradient differ- 
ential inclusion of a convex function," we know that solutions exist and are unique. Consequently, 
the notions of weakly nonincreasing and strongly nonincreasing function coincide. Therefore, it 
suffices to show that / is weakly nonincreasing on M'^ for the gradient differential inclusion 



For all C £ dpf{x), there exists v = —( G —dpf{x) such that Q^v = — HClli ^ 0- Hence, 

C^Qpffix) < for ah x G R'^. 

Proposition 1 ) now guarantees that / is weakly nonincreasing on M.'^. Since the solutions of the 
gradient differential inclusion (j62p are unique, t f{x{t)) is monotonically nonincreasing for all 
solutions 1 1— > x{t). 

The application of the Lyapunov stability theorem and the invariance principle in theorems [3] 
and m respectively, gives rise to the following nonsmooth counterpart of the classical smooth result 



for a gradient flow 56(] 



Proposition 14. Let f : M. be convex. Then, the strict minimizers of f are 

strongly stable equilibria of the gradient differential inclusion (j62p associated with f . Furthermore, if 
the level sets of f are bounded, then the solutions of the gradient differential inclusion asymptotically 
converge to the set of minimizers of f. 



Convergence rate estimates of (j62p can be found in 63l |. 

Stabilization of control systems 

Consider the control system on M'^ given by 

x = X{x,u), (77) 

where X : M'^xM"* — > M"'. The system (j77p is locally (respectively, globally) continuously stabilizable 
if there exists a continuous map k ^ M"* such that the closed-loop system 

X = X{x, k{x)) 

is locally (respectively globally) asymptotically stable at the origin. The following result by Brock- 



ett [131], see also [1^, UM^ states that a large class of control systems are not stabilizable by a 



continuous feedback controller. 

Theorem 5. Let X : x M."^ M*^ be continuous and X{0, 0) = 0. // there exists a 
continuous stabilizer of the control system (j77p . then there exists a neighborhood of the origin in 
■^d ^ j^m yjfiQg^ image by X is a neighborhood of the origin in . 

In particular, Theorem [5] implies that control systems of the form 

X = uiXi{x) H h UmXm{x), (78) 

with m < n and Xi : ^ W^, i G {!,..., m}, continuous with rank(Xi(0), . . . , X,„(0)) = m, 
cannot be stabilized by a continuous feedback controller. 
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Example 21: Nonholonomic integrator 



Consider the nonholonomic integrator [6j] 



xi = ui, (79) 
X2 = U2, (80) 
±3 = XiU2 - X2U1, (81) 

where {ui,U2) E M?. The nonholonomic integrator is a control system of the form (j78p . with 
m = 2, Xi(xi, a;2, X3) = (1, 0, —X2), and X2(2;i, 2:2, 3:3) = (0, l,xi). The nonholonomic integrator is 
controllable, that is, for each pair of states x,y £ R^, there exists an input t 1— > u{t) such that the 
corresponding solution of (|79p - (|8ip starting at x reaches y. However, the nonholonomic integrator 
does not satisfy the condition in Theorem [5l and thus is not continuously stabilizable. Indeed, 
no point of the form (0,0, e), where e ^ 0, belongs to the image of the map X : x — > M^, 

X{xi,X2,Xs, Ui,U2) = iui,U2, X1U2 - X2U1). ■ 

The condition in Theorem [5] is necessary but not sufficient. There exist control systems that 
satisfy the condition, and still cannot be stabilized by means of a continuous stabilizer. The cart 
on a circle in Example 1201 is one of these systems. The map {{xi, X2),u) — > X{xi,X2,u) satisfies the 



necessary condition in Theorem [5] but cannot be stabilized with a continuous A: : R, see 251 ]. 



Another obstruction to the existence of continuous stabilizing controllers is given by Milnor's 



theorem |12l ] , which states that the domain of attraction of an asymptotically stable equilibrium of 
a locally Lipschitz vector field must be diffeomorphic to Euclidean space. Since environments with 
obstacles are not diffeomorphic to Euclidean space, one can use Milnor's theorem to justify [2H] the 
nonexistence of continuous globally stabilizing controllers in environments with obstacles. 

The obstruction to the existence of continuous stabilizers has motivated the search for time- 
varying and discontinuous feedback stabilizers. Regarding the latter, given a discontinuous k : 
R"^ —>■ R"^, the immediate question that arises is how to understand the solutions of the resulting 
discontinuous dynamical system x = X{x,k{x)). From the previous discussion, we know that 
Caratheodory solutions are not a good candidate, since in many situations they fail to exist. The 
following result 21, [i^] shows that Filippov solutions are also not a good candidate. 



Theorem 6. Let X ■.M.'^ x R"^ R'^ be continuous and X{0,0) = 0. Assume that, for 
each U C R'" and each x E R*^, X{x,coU) = coX{x,U) holds. With solutions understood in the 
Filippov sense, if there exists a measurable, locally bounded stabilizer of the control system (I77p . 
then there exists a neighborhood of the origin in x R'" whose image by X is a neighborhood of 
the origin in R'^. 



In particular, control systems of the form (j78p cannot be stabilized by means of a discontinuous 
feedback if solutions are understood in the Filippov sense. This impossibility result, however, can 
be overcome if solutions are understood in the sample-and-hold sense, as shown in 14 1| . Let us 
briefly discuss this result in the light of the above exposition. For more details, see |24l . 25, [2^ . 
Consider the differential inclusion (135p associated with the control system (j77p . The system (j77p 



is (open loop) globally asymptotically controllable (to the origin) if is a Lyapunov weakly stable 
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equilibrium of the differential inclusion (|35p . and every point y G M'^ has the property that there 
exists a solution of (f35|) satisfying x(0) = y and limj^oo = 0. On the other hand, a map 
k : ^ stabilizes the system (|77p in i/ie sample- and-hold sense if, for all G M*^ and all 
e G (0,00), there exist 5,T £ (0, 00) such that, for all partitions vr of [0,ti] with diam(7r) < 6, the 
corresponding vr-solution t x{t) of ([77]) starting at xq satisfies ||2;(t)||2 < e for all t>T. 



The following result [4]| states that both notions, global asymptotic controllability and the exis- 



tence of a feedback stabilizer in the sample-and-hold sense, are equivalent. 

Theorem 7. Let X : M'^ x M™^ R'^ be continuous and X{0,0) = 0. Then, the control 
system ()77p is globally asymptotically controllable if and only if it admits a measurable, locally 
bounded stabilizer in the sample-and-hold sense. 

The "only if" implication is clear. The converse implication is proved by explicit construction 
of the stabilizer, and is based on the fact that the control system ()77p is globally asymptotically 



controllable if and only if it admits a continuous Lyapunov pair, see (651 ]. Using the continuous 
Lyapunov function provided by this characterization, the discontinuous feedback for the control 
system (|77l) can be constructed explicitly 2^, 41]. 



The existence of a Lyapunov pair in the sense of generalized gradients (that is, when the set- 
valued Lie derivative involving the generalized gradient is used instead of the lower set-valued 
Lie derivative involving the proximal subdifferential) , however, turns out to be equivalent to the 
existence of a stabilizing feedback in the sense of Filippov, see [66|. 



Example 1201 revisited: Cart stabilization by discontinuous feedback 

As an illustration, consider Example [20l We have already shown that (0, 0) is a globally weakly 
asymptotically stable equilibrium of the differential inclusion (j74p associated with the control sys- 
tem. Therefore, the control system is globally asymptotically controllable, and can be stabilized 
in the sample-and-hold sense by means of a discontinuous feedback. The stabilizing feedback that 



results from the proof of Theorem [7] can be described as follows, see |25|, |67l]. If placed to the left of 
the X2 axis, move in the direction of the vector field g. If placed to the right of the X2 axis, move in 
the opposite direction of the vector field g. Finally, on the X2-axis, choose an arbitrary direction. 
The stabilizing nature of this feedback can be graphically checked in Figure fT^ a) and (b). ■ 

Remarkably, for a system that is affine in the controls, it is possible to show j6^ that there exists 
a stabilizing feedback controller whose discontinuities form a set of measure zero, and, moreover, 
the discontinuity set is repulsive for the solutions of the closed-loop system. In particular, this fact 
means that, for the closed-loop system, the solutions can be understood in the Caratheodory sense. 
This situation exactly corresponds to the situation in Example [20l 



46 



Conclusions 



This article has presented an introductory tutorial on discontinuous dynamical systems. Various 
examples illustrate the pertinence of the continuity and Lipschitzness properties that guarantee 
the existence and uniqueness of classical solutions to ordinary differential equations. The lack of 
these properties in examples drawn from various disciplines motivates the need for more general 
notions than the classical one. This observation is the starting point into the three main themes 
of our discussion. First, we introduced notions of solution for discontinuous systems. Second, 
we reviewed the available tools from nonsmooth analysis to study the gradient information of 
candidate Lyapunov functions. And, third, we presented nonsmooth stability tools to characterize 
the asymptotic behavior of solutions. 

The physical significance of the solution notions discussed in this article depend on the specific 
setting. Caratheodory solutions are employed for time-dependent vector fields that depend discon- 
tinuously on time, such as dynamical systems involving impulses as well as control systems with 
discontinuous open- loop inputs. Filippov solutions are used in problems involving electrical circuits 
with switches, relay control, friction, and sliding. This observation is also valid for solution notions 
similar to Filippov's, such as Krasovskii and Sentis solutions, see "Additional Solution Notions for 
Discontinuous Systems." The notion of a vr-solution for control systems has the physical interpre- 
tation of iteratively evaluating the input at the current state and holding it steady for some time 
while the closed-loop dynamical system evolves. As illustrated above, this solution notion plays a 
pivotal role in the stabilization of asymptotically controllable systems. 

There are numerous important issues that are not treated here; "Additional Topics on Discontin- 
uous Systems and Differential Inclusions" lists some of them. The topic of discontinuous dynamical 
systems is vast, and our focus on the above-mentioned themes is aimed at providing a coherent 
exposition. We hope that this tutorial serves as a guided motivation for the reader to further ex- 
plore the exciting topic of discontinuous systems. The list of references of this manuscript provides 
a good starting point to undertake this endeavor. 
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u 




X 



Figure 1. Obstruction to the existence of continuous state-dependent feedbacl< |12l ]. The 
shaded areas represent the values of the control u that are needed to stabilize the equilibrium 
point of system ([1]). There does not exist a function x u{x) defined on M that at the same 
time is continuous and belongs to the shaded areas. 
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(a) 



(b) 



Figure 2. Brick sliding on a frictional ramp, (a) shows the physical quantities used to describe 
the example, (b) shows the one-dimensional phase portraits of ([2]) corresponding to values of 
the friction coefficient v between and 2, with a ramp inclination of 30 degrees. The phase 
portrait shows that, for sufficiently small values of v, every trajectory that starts with positive 
initial velocity ("moving to the right") never stops. However, for sufficiently large values 
of V, every trajectory that starts with positive initial velocity eventually stops and remains 
stopped. However, there is no continuously differentiable solution of ([2]) that exhibits this type 
of behavior. We thus need to expand our notion of solution beyond continuously differentiable 
solutions by understanding the effect of the discontinuity in ([2]). 
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(a) 



Xl 



(b) 



Figure 3. Nonsmooth harmonic oscillator, (a) shows the phase portrait on [—1, 1]^ of the vector 
field {xi,X2) ^ (a;2, — sign(xi)), while (b) shows the contour plot on [—1,1]^ of the function 
{xi,X2) I— > + The discontinuity of the vector field along the a;2-coordinate axis makes 
it impossible to find continuously differentiable solutions of ([TO]) . On the other hand, the level 
sets in (b) match the phase portrait in (a) everywhere except for the X2-coordinate axis, which 
suggests that trajectories along the level sets are candidates for solutions of (fTO|) in a sense 
different from the classical one. 
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(a) (b) 

Figure 4. Move-away-from-nearest-neighbor interaction law. A node in (a) computes different 
directions of motion if placed slightly to the right or to the left of the bisector line defined 
by two polygonal boundaries. A node in (b) computes different directions of motion if placed 
slightly to the right or to the left of the half plane defined by two other nodes. In both cases, 
the vector field is discontinuous. 
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(a) (b) (c) (d) 

Figure 5. Discontinuous and not-one-sided Lipschitz vector fields. The vector fields in (a) 
and (b), which differ only in their values at 0, are discontinuous, and thus do not satisfy the 
hypotheses of Proposition [TJ Therefore, the existence of solutions is not guaranteed. In fact, 
the vector field in (a) has no solution starting from 0, whereas the vector field in (b) has a 
solution starting from all initial conditions. The vector fields in (c) and (d) are neither locally 
Lipschitz nor one-sided Lipschitz, and thus do not satisfy the hypotheses of Proposition [2l 
Therefore, uniqueness of solutions is not guaranteed. The vector field in (c) has two solutions 
starting from 0. However, the vector field in (d) has a unique solution starting from all initial 
conditions. 
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Xi 



(a) 







(b) 



Figure 6. Move-away-from-nearest-neighbor interaction law for one agent moving in the square 
[—1,1]^ C M?. (a) shows the phase portrait, while (b) shows several Filippov solutions. On the 
diagonals, the vector field pushes trajectories out, whereas, outside the diagonals, the vector 
field pushes trajectories in. This fact makes it impossible to construct a Caratheodory solution 
starting from any initial condition on the diagonals of the square. 
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Figure 7. Illustration of the notion of sample-and-hold solution. For the control system x = u, 
we choose the control input u{x) = x. The upper curve is the classical solution starting from 
xq = 1, while the lower curve is the sample-and-hold solution starting from xq = 1 corresponding 



to the TT-partition {0, j, \, f , 1} of [0, 1]. 
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(c) (d) 



Figure 8. Piecewise continuous vector fields. These dynamical systems are continuous on Vi 
and P2, and discontinuous on Sx- In (a) and (b), Filippov solutions, known as transversally 
crossing trajectories, cross Sx- In (c), there are two Filippov solutions, known as repulsive 
trajectories, starting from each point in Sx- Finally, in (d), the Filippov solutions that reach 
Sx, known as attractive trajectories, continue sliding along Sx- 
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-1 -.5 .5 1 

Figure 9. Generalized gradient vector field. This plot shows the generalized gradient vector 
field of the minimum-distance-to-polygonal-boundary function smg : Q ^ M on the square 
[—1,1]^. The vector field is discontinuous on the diagonals of the square. Note the similarity 
with the phase portrait of the move-away-from-nearest-neighbor interaction law for one agent 
moving in the square [—1,1]^ C M? plotted in Figure [51 
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(a) (b) 



Figure 10. Geometric interpretations of the proximal subdifferential of the function x |x| at 
X = computed in Example 1171 The epigraph of the function is shaded, while the proximal 
normal cone to the epigraph at is striped. In (a), according to ([5T]) . each proximal subgradient 
corresponds to a direction tangent to a parabola that fits under the epigraph of the function. In 
(b), according to ([52]) . each proximal subgradient can be uniquely associated with an element 
of the proximal normal cone to the epigraph of the function. 
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Xl 



(a) 



Xl 



(b) 



Figure 11. Nonsmooth harmonic oscillator with dissipation, (a) shows the phase portrait 
on [—1,1]^ of the vector field (xi,X2) ^ (x2, — sign(xi) — /csign(x2)), where k = 0.75, while 
(b) shows some Filippov solutions of the associated dynamical system (fTO|) . In the nonsmooth 
harmonic oscillator in Example[3l the equilibrium at the origin is strongly stable, but not strongly 
asymptotically stable, see Figure [3l The addition of dissipation renders the equilibrium at the 
origin strongly asymptotically stable. 
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Figure 12. From left to right, evolution of the nonsmooth gradient flow of the function — smg in 
a convex polygon. At each snapshot, the value of smg is the radius of the largest disk (plotted 
in gray) contained in the polygon with center at the current location. The flow converges in 
finite time to the incenter set, which, for this polygon, is a singleton whose only element is 
the center of the disk in the rightmost snapshot. 
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(a) (b) (c) 

Figure 13. The move-away-from-nearest-neighbor interaction law for solving a sphere-packing 
problem within the polygon Q. (a) is the initial configuration, (b) is the evolution, and (c) is 
the final configuration of a Filippov solution. In (c), the minimum radius of the shaded spheres 
corresponds to the value of the locally Lipschitz function Tisp, defined in (169]) . The Filippov 
solutions of the move-away-from-nearest-neighbor dynamical system monotonically increase 
the value of Tisp- In (c), every node is at equilibrium since the infinitesimal motion of a node 
in any direction would place it closer to at least another node. This discontinuous dynamical 
system is an example of how simple local interactions can achieve a global objective. 
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(a) (b) (c) 

Figure 14. Cart on a circle, (a) shows the phase portrait of the input vector field {xi,X2) ^ 
{xf — 2x1x2), (b) shows its integral curves, and (c) shows the contour plot of the function 

/ (xi,X2) ^ , ^^^^'^ , (0,0) ^ 0. The origin cannot be asymptotically stabilized by 

^^xl+x^ + \x-^\ 

means of continuous feedback in the cart dynamics ([72|) - ([73]) . However, the origin can be 
asymptotically stabilized by means of discontinuous feedback when solutions are understood 
in the sample-and-hold sense. 
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Sidebar 1: Solutions with Jumps 

In this article, we focus entirely on absolutely continuous solutions of ordinary differential equations. 
However, nonsmooth continuous-time systems can possess discontinuous solutions that admit jumps 
in the state. Such notions are appropriate for dealing with, for instance, mechanical systems subject 
to unilateral constraints [15|]. As an example, a bouncing ball hitting the ground experiences an 
instantaneous change of velocity. This change corresponds to a discontinuous jump in the trajecton 
describing the evolution of the velocity 



of the ball. Both measure differential inclusions 

S4 . S5 . [s^ are approach es t hat specifically allow for 



and linear complementarity systems [S 

discontinuous solutions. Within hybrid systems theory [S7l . IS8l . [S 
arise in systems that involve both continuous- and discrete-time evolutions. 



S10|, discontinuous solutions 
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Sidebar 2: Additional Solution Notions for Discontinuous Systems 

Several solution notions are available in addition to Caratheodory, Filippov, and s amp l e-an d-hold 
solutio ns. T hese notions include the ones considered by K rasovskii 34 1, Hermes Sll . S12| . Am- 
brosio [S13l |. Sentis [35j], and Yakub ovich-Leonov- G elig jS14| . see Table ISTl As in the case in which 



the vector field is continuous, Euler solutions [18l . 1241 ] are useful for establishing existence and in 
characterizing basic mathematical properties of th e dy namical system. Additional notions of so- 
lutions for discontinuous systems are provided in S14l . Section 1.1.3]. With so many notions of 
solution available, various works explore the relationships among them. For example, Caratheodory 
an d Fil ippov solutions are compared in [S15l | ; Caratheodory and Krasovskii solutions are compared 
in ISldl : Caratheodory, Euler, sample-and-hold, Filippov, and Krasovskii solutions are compared 
in [ld|: Hermes. Filippov, and Krasovs kii s olutions are compared in S12| : and Caratheodory, Euler, 
and Sentis solutions are compared in [S17| |. 



Notion of solution 


References 


Caratheodory 


[18j 


Filippov 


[18] 


Krasovskii 


m 


Euler 


[18, 2^ 


Sample-and-hold 


[23] 


Hermes 


fSll. S121 


Sentis 


[35], [Sia 


Ambrosio 


[S13] 


Yakubovich-Leonov-Gelig 


[Sl^ 



Table SI. Several notions of solution for discontinuous dynamics. Depending on the specific 
problem, some notions give more physically meaningful solution trajectories than others. 
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Sidebar 3: Additional Topics on Discontinuous Systems and Differential Inclusions 

Beyond the topics discussed in this article, we briefly mention several that are relevant to sys- 
tems and control. These topics include continuous dependence of solutions with respect to initial 

j Jigt . robustness properties against external disturbances and state 
S18| , condi tions for t he existence of periodic solutions 



conditions and parameters 
measurement errors (j^ ] and 

tions [3^, converse Lyapunov theorems S19 . |S20 |. controllability of differential inclusions |2C 



Ti,^, bifurca- 



output trackin g in differential inclusions [43|], systems subject to gradient nonlinearities |S14l ]. vi- 
ability theory S2l\\ , ma ximal monotone inclusions [19j], projected dynamical systems and vari- 



ational i nequalitie s |S22[| . and numerical methods for discontinuous systems and differential in- 
clusions S23l . |S24| . |S25| | . Several works report equivalence results among different approaches to 
nonsmooth systems, including [19|] on the equivalence between differential inclusions and projected 
dynamical systems, and jS26l] on the equivalence of these formalisms with complementarity systems. 
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Sidebar 4: Index of Symbols 

The following is a list of the symbols used throughout the article. 



Symbol : Description and page(s) when applicable 

G[X] : Set-valued map associated with a control system X : M*^ x ^ — > M"^ 

co(>S') : Convex closure of a set S C R'^ 

co(S') : Convex hull of a set 5 C R'^ 

diam(7r) : Diameter of the partition vr 

f°{x;v) : Generalized directional derivative of the function / : M'^ — > M at x € M"^ in the 

direction of f S M*^ 

f'{x;v) : Right directional derivative of the function / : M'^ — > M at a; G M*^ in the 

direction of f E M*^ 

Sx '■ Set of points where the vector field X : is discontinuous 

dist(p, 5) : Euclidean distance from the point p G M'^ to the set C R'^ 

F[X] : Filippov set- valued map associated with a vector field X : M'' ^ M'^ 

df : Generalized gradient of the locally Lipschitz function / : M'^ ^ M 

V/ : Gradient of the differentiable function / : R'^ ^ M 

Ln(S') : Least-norm elements in the closure of the set 5 C M'^ 

Q{x) : Set of limit points of a curve t ^ x{t) 

J\f : Nearest-neighbor map 

Ue : Unit normal to the edge e of a polygon Q pointing toward the interior of Q 

Q.f : Set of points where the locally Lipschitz function / : — > M fails to be 

differentiable 
vr : Partition of a closed interval 

55(5') : Set whose elements are all the possible subsets of S" C M*^ 

dpf : Proximal sub differential of the lower semicontinuous function / : M'^ ^ R 

Cj^f : Set-valued Lie derivative of the locally Lipschitz function / : M'^ ^ M with 

respect to the set-valued map ^ 55 (M'') 

Cxf '■ Set-valued Lie derivative of the locally Lipschitz function / : M'^ — > M with 

respect to the Filippov set-valued map F[X] : «B(M'^) 
Cjrf : Lower set-valued Lie derivative of the lower semicontinuous function / : M'^ ^ M 

with respect to the set-valued map 
Ljrj : Upper set-valued Lie derivative of the lower semicontinuous function / : R"^ — > 

R with respect to the set-valued map : R"^ ^ «B(R'^) 
: Set-valued map 

smQ : Minimum distance function from a point in a convex polygon Q C R*^ to the 

boundary of Q 
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Sidebar 5: Locally Lipschitz Functions 

A function / : M'^ — > M™" is locally Lipschitz at x (^M.'^ if tliere exist Lx,£ €z (0, oo) such that 

\\f{y)- f{y)h<L4y-y'\\2, 

for all y,y' G B{x,e). A function that is locally Lipschitz at x is continuous at x, but the converse 
is not true. For example, / : M ^ M, /(x) = y/\x\, is continuous at 0, but not locally Lipschitz 
at 0, see Figure ISTl f a) . A function is locally Lipschitz on S" C R'^ if it is locally Lipschitz at x for 



(a) 



(b) 



Figure SI. Illustration of the difference among continuous, locally Lipschitz, and differentiable 
functions, (a) shows the graph of / : M ^ M, f{x) = \/\x\, which is continuous at 0, but not 
locally Lipschitz at 0. (b) shows the graph of / : M ^ M, f{x) = \x\, which is is locally Lipschitz 
at 0, but not differentiable at 0. 



all X G 5. If / is loc ally L ipschitz on W^, we simply say / is locally Lipschitz. Convex functions 
are locally Lipschitz jS27| . and hence concave functions are also locally Lipschitz. Note that a 
function that is continuously differentiable at x is locally Lipschitz at x, but the converse is not 
true. For example, / : R — > M, f{x) = \x\, is locally Lipschitz at 0, but not differentiable at 0, see 
Figure ISTT b). A function / : R x R*^ — > R™ that depends explicitly on time is locally Lipschitz at 
X gW^ if there exists e G (0, oo) and : R ^ (0,oo) such that \\f{t,y)-f{t,y')\\2 < Lx{t)\\y-y'\\2 
for alH G R and y, y' G B{x, e). 
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Sidebar 6: Caratheodory Conditions for Time-varying Vector Fields 



Consider the differential equation 

xit)=X{t,x{t)), (SI) 

where X : [0, oo) x M*^ — > is a time-varying vector field. The following resul t is taken from [l 
a weaker version of the Caratheodory conditions is given, for instance, in S2^. 



Proposition SI. Let X : [0, oo) x M*^ — > M'^. Assume that (i) for almost all t G [0, oo), 
the map x i— > X{t,x) is continuous, (ii) for each x G W^, the map t ^ X{t,x) is measurable, and 
(Hi) X is locally essentially hounded, that is, for all {t,x) G [0, oo) x M"^, there exist e G (0, oo) and 
an integrahle function m : \t,t + S\ — > (0, oo) such that \\X{s, y)\\2 < m{s) for almost all s G [t, t + (^] 
and ally G B{x,£). Then, for all (tQ,xo) G [0, oo) xM.'^, there exists a Caratheodory solution of (IS1|) 
with initial condition x{tQ) = xq. 

The specialization of Proposition [ST] to a time-invariant vector field requires that the vector field 
be continuous, which in turn guarantees the existence of a classical solution. 
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Sidebar 7: Set-valued Maps 

A set-valued map, as its name suggests, is a map that assigns sets to points. We consider time- 
varying set-valued maps of the form T : [0, oo) x M*^ — > *B(M'^). Recall that *B(M'^) denotes the 
collection of all subsets of W^. The map assigns to each point {t, x) G [0, oo) x the set 
T{t, x) C R'^. Note that a standard map / : [0, oo) X ^ can be interpreted as a singleton- 
valued map. A complete analysis for set-valued maps can be developed, as in the case of standard 
maps (i^. Here, we are mainly interested in concepts related to boundedness and continuity, which 
we define next. 

The set-valued map J- : [0, oo) x M"' — > *B(M'^) is locally bounded (respectively, locally essentially 
bounded) at (t, x) £ [0, oo) x M*^ if there exist e,6 £ (0, oo) and an integrable function m : [1,1 + 6] 
(0,oo) such that ||z||2 < m(s) for all z £ J^{s,y), all s £ [t,t + 6], and all y £ B{x,e) (respectively, 
almost all y £ B{x,£) in the sense of Lebesgue measure). 

The time-invariant set- valued map .7^ : M'^ — > 53 (M"^) is upper semicontinuous (respectively, lower 
semicontinuous) at a; G M*^ if, for all e £ (0,oo), there exists 6 £ (0, oo) such that J^{y) ^ J^{x) + 
B{0,e) (respectively, T{x) C jr(y) + B{0,e)) for all y £ B{x,6). The set-valued map : ^ 
!B(M'^) is continuous at x £ if it is both upper and lower semicontinuous at x £ W^. Finally, 
the set- valued map J- : M*^ — > 53 (M*^) is locally Lipschitz at x G M'^ if there exist Lx,£ £ (0, oo) such 
that 

J'iy') ^ J^{y) + L^Wy - y'\\^B{0,l), 

for all y,y' £ B{x,£). A locally Lipschitz set-valued map at x is upper semicontinuous at x, but 
the converse is not true. 

The notion of upper semicontinuity of a map / : R'^ — > M, which is defined in the section "The 
proximal sub differential of a lower semicontinuous function," is weaker than the notion of upper 
semicontinuity of / when viewed as a (singleton-valued) set-valued map from to !B(M). Indeed, 
the latter is equivalent to the condition that / : M'^ ^ M is continuous. 
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Sidebar 8: Caratheodory Solutions of Differential Inclusions 



A differential inclusion [l9|, l20|] is a generalization of a differential equation. At each state, a 
differential inclusion specifies a set of possible evolutions, rather than a single one. This object is 
defined by means of a set-valued map, see "Set-valued Maps." The differential inclusion associated 
with a time-varying set-valued map T : [0, oo) xW^ ^ *B(M'^) is an equation of the form 

x{t) (^T{t,x{t)). (SI) 

The point Xf. S is an equilibrium of the differential inclusion if G J-{t,Xe) for all t G [0,oo). 
We now define the notion of solution of a differential inclusion in the sense of Caratheodory. 

A Caratheodory solution of (|Sip defined on [ic^i] C [0, co) is an absolutely continuous map 
X : [io,ii] — > I^"' such that x{t) G J^{t,x{t)) for almost every t G [ioi^i]- The existence of at 
least one solution starting from each initial condition is guaranteed by the following result (see, for 
instance, 



14,^ 



Proposition SI. Let T : [0, oo) x M'^ ^ 53(R'^) he locally bounded and take nonempty, 
compact, and convex values. Assume that, for each t £M, the set-valued map x ^ J'{t, x) is upper 
semicontinuous, and, for each x G M"^, the set-valued map 1 1-^ T{t, x) is measurable. Then, for all 
{to,XQ) G [0, oo) xM"', there exists a Caratheodory solution of (jSip with initial condition x(to) = xq. 



This result is sufficient for our purposes. Additional existence results based on alternative as- 
sumptions are given in The uniqueness of Caratheodory solutions is guaranteed by the 
following result. 

Proposition S2. In addition to the hypotheses of Proposition lSl\ assume that, for all 
X G M"^, there exist e G (0, oo) and an integrable function Lx ^ (0, oo) such that 

{v-wf{y-y')<Lx{t)\\y-y'\\l (S2) 

for almost every y,y' G B{x,e), every t G [0, oo), every v G !F{t,y), and every w G J^{t,y'). 
Then, for all {to,xo) G [0, oo) x W^, there exists a unique Caratheodory solution of (ISip with initial 
condition x{t()) = xq. 



The following example illustrates propositions [ST] and [S2l Following |43|] , consider the set- valued 
map : M ^ 5S(]R) defined by 



Tix) 



0, X / 0, 

[-1,1], x = 0. 



Note that is upper semicontinuous, but not lower semicontinuous, and thus it is not continuous. 
This set-valued map satisfies all of the hypotheses in Proposition ISll and therefore Caratheodory 
solutions exist starting from all initial conditions. In addition, satisfies ()S2p as long as y and y' 
are nonzero. Therefore, Proposition IS2I guarantees the uniqueness of Caratheodory solutions. In 
fact, for every initial condition, the Caratheodory solution of x{t) G J^{x{t)) is just the equilibrium 
solution. 
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Sidebar 9: Uniqueness of Filippov Solutions of Piecewise Continuous Vector Fields 

Here we justify why, in order to guarantee the uniqueness of Filippov solutions for a piecewise 
continuous vector field, we cannot resort to Proposition U] and instead must use Proposition [5j 
To see this, consider a piecewise continuous vector field X : — > M*^, d > 2, and let x G Sx 
be a point of discontinuity. Let us show that X is not essentially one-sided Lipschitz on every 
neighborhood of x. For simplicity, assume x belongs to the boundaries of just two sets, that is, 
X G bndry(P,j) n bndry(I?j) (the argument proceeds similarly for the general case). For e > 0, we 
show that p2p is violated on a set of nonzero measure contained in B{x,e). Notice that 

{X{y) - X{y')f{y - y') = \\X{y) - X(y')||2 \\y - v'h cos y'), 

where a{y, y') = Z{X{y) — X{y'),y — y') is the angle between the vectors X{y) — X{y') and y — y' . 
Therefore, (i32|l is equivalent to 

\\X{y) - X{y')\\2 cosa(y, y') < L\\y - y'h- (S3) 

Consider the vectors X|^(a;) and X^jy{x). Since X is discontinuous at x, we have X^jy{x) / 
X^-^{x). Take y £ ViCi B{x, e) and y' S Vj n B{x, e). Note that, as y and y' tend to x, the vector 
X{y) — X{y') tends to X^^yXx) — X^jy{x). Consider then the straight line i that crosses Sx-, passes 
through X, and forms a small angle /? > with X^^^x) — ^.^(x), see Figure [S2l 




Figure S2. Piecewise continuous vector field. Tlie vector field has a unique Filippov solution 
starting from all initial conditions. Note that solutions that reach Sx coming from Vj cross it, 
and then continue in Vi. However, the vector field is not essentially one-sided Lipschitz, and 
hence Proposition |4] cannot be invoked to conclude uniqueness. 

Let R be the set enclosed by the line ^ and the line in the direction of the vector X|^(x) — X|^(x). 
If y G n ii and y' £ Vj n R tend to x, we deduce that \\y — y'\\2 while, at the same time, 

\\X{y) - X{y')\\2 I cosa{y,y')\ 

> \\X{y) - X{y')\\2 cos/3 \\X^wS^) - X|^(x)||2 cos/3 > 0. 

Therefore, there does not exist L £ (0, oo) such that ([S3]) is satisfied for y G iJnPj H B{x,£) and 
y' £ RnVj D B{x,£). Thus, X is not essentially one-sided Lipschitz on any neighborhood of x, 
and the hypotheses of Proposition d] do not hold. 
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Sidebar 10: Regular Functions 

To introduce the notion of regular function, we need to first define the right directional derivative 
and the generalized right directional derivative. Given / : M*^ ^ M, the right directional derivative 
of / at X in the direction of f G M'^ is defined as 

.// ^ 1- f{x + hv) - f{x) 
J [x; V) = lim 



h^0+ h 

when this limits exists. On the other hand, the generalized directional derivative of / at a; in the 
direction of f G M*^ is defined as 

.Of , y f{y + hv)-f{y) f{y + hv)-f{y) 
J (x; f ) = limsup = lim sup . 

y-*x h i^0+y^B(x,5) h 

The advantage of the generalized directional derivative compared to the right directional derivative 
is that the limit always exists. When the right directional derivative exists, these quantities may be 
different. When they are equal, we call the function regular. More formally, a function / : — > M 
is regular at x ^ M.'^ if, for all v S R'^, the right directional derivative of / at x in the direction of 
V exists, and f'{x;v) = f°{x;v). A function that is continuously differentiable at x is regular at x. 
Also, a convex function is regular (cf. Proposition 2.3.6]). 

The function g : M — > M, g{x) = — |x|, is not regular. Since g is continuously differentiable 
everywhere except for zero, it is regular on IR\{0}. However, its directional derivatives 

g{0;v) = < g {0;v) = 

[ V, V <0, 

do not coincide. Hence, g is not regular at 0. 
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